Difference between revisions of "Nemenman et al., 2002"

From Ilya Nemenman: Theoretical Biophysics @ Emory
Jump to: navigation, search
nemenman>Ilya
 
m (1 revision imported)
 
(No difference)

Latest revision as of 12:28, 4 July 2018

Back to the full Publications list.

I Nemenman, F Shafee, and W Bialek. "Entropy and inference, revisited." In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Adv. Neural Inf. Proc. Syst. 14, Cambridge, MA, 2002. MIT Press. PDF, arXiv.

Abstract
We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.