On signal representations within the Bayes decision framework

Abstract

This work presents new results in the context of minimum probability of error signal representation (MPE-SR) within the Bayes decision framework. These results justify addressing the MPE-SR criterion as a complexity-regularized optimization problem, demonstrating the empirically well understood tradeoff between signal representation quality and learning complexity. Contributions are presented in three folds. First, the stipulation of conditions that guarantee a formal tradeoff between approximation and estimation errors under sequence of embedded transformations are provided. Second, the use of this tradeoff to formulate the MPE-SR as a complexity regularized optimization problem, and an approach to address this oracle criterion in practice is given. Finally, formal connections are provided between the MPE-SR criterion and two emblematic feature transformation techniques used in pattern recognition: the optimal quantization problem of classification trees (CART tree pruning algorithms), and some versions of Fisher linear discriminant analysis (LDA).