Nemenman et al., 2002

From Ilya Nemenman: Theoretical Biophysics @ Emory
Revision as of 02:43, 18 November 2006 by nemenman>Ilya
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Back to the full Publications list.

I Nemenman, F Shafee, and W Bialek. "Entropy and inference, revisited." In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Adv. Neural Inf. Proc. Syst. 14, Cambridge, MA, 2002. MIT Press. PDF, arXiv.

We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.