Monthly
288 pp. per issue, 6 x 9,
illustrated
Founded: 1989
ISSN 0899-7667
E-ISSN 1530-888X
2008 ISI Impact Factor: 2.378
|
March 1994, Vol. 6, No. 2, Pages 334-340
Posted Online April 10, 2008.
(doi:10.1162/neco.1994.6.2.334)
© 1994 Massachusetts Institute of Technology
Statistical Physics, Mixtures of Distributions, and the EM Algorithm Alan L. YuilleDivision of Applied Sciences, Harvard University, Cambridge, MA 02138 USA Paul StolorzJet Propulsion Laboratory, MS 198-219, Pasadena, CA 91109 and Santa Fe Institute, Santa Fe, NM 87501 USA Joachim UtansInternational Computer Science Institute, 1947 Center Street, Suite 600, Berkeley, CA 94704 USA We show that there are strong relationships between approaches to optmization and learning based on statistical physics or mixtures of experts. In particular, the EM algorithm can be interpreted as converging either to a local maximum of the mixtures model or to a saddle point solution to the statistical physics system. An advantage of the statistical physics approach is that it naturally gives rise to a heuristic continuation method, deterministic annealing, for finding good solutions. Cited byBehrooz Safarinejadian, Mohammad B. Menhaj, Mehdi Karrari. (2009) A distributed EM algorithm to estimate the parameters of a finite mixture of components. Knowledge and Information Systems Online publication date: 4-Jul-2009. CrossRef F. Wang, B.C. Vemuri, A. Rangarajan, S.J. Eisenschenk. (2008) Simultaneous Nonrigid Registration of Multiple Point Sets and Atlas Construction. IEEE Transactions on Pattern Analysis and Machine Intelligence 30:11, 2011-2022 Online publication date: 1-Dec-2008. CrossRef Qi Zhao, David J. Miller. (2005) Mixture Modeling with Pairwise, Instance-Level Class Constraints. Neural Computation 17:11, 2482-2507 Online publication date: 1-Nov-2005. Abstract
| PDF (339 KB)
| PDF Plus (346 KB) Carlos Ordonez, Edward Omiecinski. (2005) Accelerating EM clustering to find high-quality solutions. Knowledge and Information Systems 7:2, 135-157 Online publication date: 1-Mar-2005. CrossRef S. Cang, H. Yu. (2005) Novel probability neural network. IEE Proceedings - Vision, Image, and Signal Processing 152:5, 535 Online publication date: 1-Feb-2005. CrossRef Akihiro Minagawa, Yukihiko Kobayashi, Norio Tagawa, Toshiyuki Tanaka. (2003) Detection of vanishing point sequence with temporal fluctuation. Systems and Computers in Japan 34:2, 1-12 Online publication date: 1-Mar-2003. CrossRef Ing-Tsung Hsiao, Anand Rangarajan, Gene Gindi. (2003) Bayesian image reconstruction for transmission tomography using deterministic annealing. Journal of Electronic Imaging 12:1, 7 Online publication date: 1-Feb-2003. CrossRef T. Heskes. (2001) Self-organizing maps, vector quantization, and mixture modeling. IEEE Transactions on Neural Networks 12:6, 1299 CrossRef Bin Luo, E.R. Hancock. (2001) Structural graph matching using the EM algorithm and singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence 23:10, 1120 CrossRef Sun-Yuan Kung, J. Taur, Shang-Hung Lin. (1999) Synergistic modeling and applications of hierarchical fuzzy neural networks. Proceedings of the IEEE 87:9, 1550 CrossRef Sun-Yuan Kung, Jenq-Neng Hwang. (1998) Neural networks for intelligent multimedia processing. Proceedings of the IEEE 86:6, 1244-1272 Online publication date: 1-Jul-1998. CrossRef Lloyd P. M. Johnston, Mark A. Kramer. (1998) Estimating state probability distributions from noisy and corrupted data. AIChE Journal 44:3, 591-602 Online publication date: 1-Apr-1998. CrossRef Thore Graepel, Matthias Burger, Klaus Obermayer. (1997) Phase transitions in stochastic self-organizing maps. Physical Review E 56:4, 3876-3890 Online publication date: 1-Nov-1997. CrossRef Martin Kloppenburg, Paul Tavan. (1997) Deterministic annealing for density estimation by multivariate normal mixtures. Physical Review E 55:3, R2089-R2092 Online publication date: 1-Apr-1997. CrossRef Akio Utsugi. (1997) Hyperparameter Selection for Self-Organizing Maps. Neural Computation 9:3, 623-635 Online publication date: 1-Mar-1997. Abstract
| PDF (244 KB)
| PDF Plus (267 KB) T. Hofmann, J.M. Buhmann. (1997) Pairwise data clustering by deterministic annealing. IEEE Transactions on Pattern Analysis and Machine Intelligence 19:1, 1 CrossRef V.P. Kumar, E.S. Manolakos. (1997) Unsupervised statistical neural networks for model-based object recognition. IEEE Transactions on Signal Processing 45:11, 2709 CrossRef S. Gold, A. Rangarajan. (1996) A graduated assignment algorithm for graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 18:4, 377-388 Online publication date: 1-May-1996. CrossRef David Miller, Kenneth Rose. (1996) Hierarchical, Unsupervised Learning with Growing via Phase Transitions. Neural Computation 8:2, 425-450 Online publication date: 15-Feb-1996. Abstract
| PDF (1168 KB)
| PDF Plus (606 KB) Lei Xu, Michael I. Jordan. (1996) On Convergence Properties of the EM Algorithm for Gaussian Mixtures. Neural Computation 8:1, 129-151 Online publication date: 1-Jan-1996. Abstract
| PDF (929 KB)
| PDF Plus (478 KB) D. Miller, A.V. Rao, K. Rose, A. Gersho. (1996) A global optimization technique for statistical classifier design. IEEE Transactions on Signal Processing 44:12, 3108 CrossRef |
RSS (TOC Alert)
PDF