Shrinkage and Denoising by Minimum Message Length

This paper examines orthonormal regression and wavelet denoising within the Minimum Message Length (MML) framework. A criterion for hard thresholding that naturally incorporates parameter shrinkage is derived from a hierarchical Bayes approach. Both parameters and hyperparameters are jointly estimated from the data directly by minimisation of a two-part message length, and the threshold implied by the new criterion is shown to have good asymptotic optimality properties with respect to zero-one loss under certain conditions. Empirical comparisons made against similar criteria derived from the Minimum Description Length principle demonstrate that the MML procedure is competitive in terms of squared-error loss.