Sunday, November 27, 2011

How biased are maximum entropy models?

From Yaroslav's blog, this is of interest to the Experimental Probabilistic Hypersurface approach which computes the probability distribution.that maximizes entropy :for difficult to compute models (read too long to run on a computer). Here is the paper: How biased are maximum entropy models? by Jakob H. Macke, Iain Murray, Peter E. Latham. The abstract reads:
Maximum entropy models have become popular statistical models in neuroscience and other areas in biology, and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data sets can be subject to sampling bias; i.e. the true entropy of the data can be severely underestimated. Here we study the sampling properties of estimates of the entropy obtained from maximum entropy models. We show that if the data is generated by a distribution that lies in the model class, the bias is equal to the number of parameters divided by twice the number of observations. However, in practice, the true distribution is usually outside the model class, and we show here that this misspecification can lead to much larger bias. We provide a perturbative approximation of the maximally expected bias when the true model is out of model class, and we illustrate our results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.

Friday, November 25, 2011

Uncertainty Quantification at the Statistical and Applied Mathematical Sciences Institute

(there is an entry on Nukt Blanche pointing to the connection with compressive sensing and uncertainty quantification )

Dr. Adrian Sandu - tutorial lecture on Data Assimilation for Uncertainty Quantification
Habib Najm's tutorial lecture on Foundations for Uncertainty Quantification
Peter Kitanidis Inverse Problem and Calibration Uncertainty Quantification tutorial lecture
Susie Bayarri's tutorial lecture on Representation and Propagation of Uncertainty
Dan Cooley: Statistics of Extremes (Tutorial talk)
Uncertainty Quantification Summer School presentation by Dr. Adrian Sandu: Variational Data Assimilation
Uncertainty Quantification Summer School presentation by Dr. Dan Cooley: Statistical Analysis of Rare Events
Uncertainty Quantification Summer School presentation by Dr. Dongbin Xiu: Sensitivity Analysis and Polynomial Chaos for Differential Equations
Uncertainty Quantification Summer School presentation by Dr. Doug Nychka: Data Assimilation and Applications in Climate Modeling
Dr. Douglas Nychka,Director of the Institute of Mathematics Applied to Geosciences for the National Center for Atmospheric Research (NCAR), spoke to an audience on February 15 about climate change.