Simple Search

Query:

Enter a query and click »Search«...

Format:

Display: entries per page entries

Zbl 0997.94009
Amari, Shun-ichi
Information geometry on hierarchy of probability distributions. (English)
[J] IEEE Trans. Inf. Theory 47, No.5, 1701-1711 (2001). ISSN 0018-9448

Summary: An exponential family or mixture family of probability distributions has a natural hierarchical structure. This paper gives an `orthogonal' decomposition of such a system based on information geometry. A typical example is the decomposition of stochastic dependency among a number of random variables. In general, they have a complex structure of dependencies. Pairwise dependency is easily represented by correlation, but it is more difficult to measure effects of pure triplewise or higher order interactions (dependencies) among these variables. Stochastic dependency is decomposed quantitatively into an `orthogonal' sum of pairwise, triplewise, and further higher order dependencies. This gives a new invariant decomposition of joint entropy. This problem is important for extracting intrinsic interactions in firing patterns of an ensemble of neurons and for estimating its functional connections. The orthogonal decomposition is given in a wide class of hierarchical structures, including both exponential and mixture families. As an example, we decompose the dependency in a higher order Markov chain into a sum of those in various lower order Markov chains.

MSC 2000:: *94A17 Measures of information
62B10 Statistical information theory
62E10 Structure theory of statistical distributions
94A15 General topics of information theory

Keywords: extended Pythagoras theorem; Kullback divergence; probability distributions; information geometry; decomposition of joint entropy; higher-order Markov chain

Comment on this Item PDF MathML XML ASCII DVI PS BibTeX Online Ordering Article Journal

	Scientific prize winners of the ICM 2010
	Overhang
	Lie groups, physics and geometry. An introduction for physicists, engineers and chemists.

Simple Search

Zentralblatt MATH Berlin [Germany]