×

Self-organizing maps: Ordering, convergence properties and energy functions. (English) Zbl 0747.92006

Summary: We investigate the convergence properties of the self-organizing feature map algorithm for a simple, but very instructive case: the formation of a topographic representation of the unit interval \([0,1]\) by a linear chain of neurons. We extend the proofs of convergence of T. Kohonen [see “Self-organization and associative memory”, 2. ed. (1988; Zbl 0659.68100)] and of M. Cottrell and J.-C. Fort [Ann. Inst. Henri Poincaré, Probab. Stat. 23, 1-20 (1987; Zbl 0605.92002)] to hold in any case where the neighborhood function, which is used to scale the change in the weight values at each neuron, is a monotonically decreasing function of distance from the winner neuron.
We prove that the learning dynamics cannot be described by a gradient descent on a single energy function, but may be described using a set of potential functions, one for each neuron, which are independently minimized following a stochastic gradient descent. We derive the correct potential functions for the one- and multi-dimensional case, and show that the energy functions given by V. V. Tolat [Biol. Cybern. 64, No. 2, 155-164 (1990; Zbl 0711.92002)] are an approximation which is no longer valid in the case of highly disordered maps or steep neighborhood functions.

MSC:

92B20 Neural networks for/in biological studies, artificial life and related topics
68T05 Learning and adaptive systems in artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Cottrell M, Fort JC (1987) Etude d’un processus d’auto-organization. Ann Inst Henri Poincaré 23:1–20 · Zbl 0605.92002
[2] Durbin R, Mitchison G (1990) A dimension reduction framework for understanding cortical maps. Nature 343:644–647 · doi:10.1038/343644a0
[3] Durbin R, Willshaw D (1987) An analogue approach to the travelling salesman problem using an elastic net method. Nature 326:689–691 · doi:10.1038/326689a0
[4] Erwin E, Obermayer K, Schulten K (1992) Self-organizing maps: Stationary states, metastability and convergence rate. Biol Cybern (this issue) · Zbl 0747.92005
[5] Favata F, Walker R (1991) A study of the application of Kohonentype neural networks to the travelling salesman problem. Biol Cybern 64:463–468 · Zbl 0722.92002 · doi:10.1007/BF00202610
[6] Geszti T (1990) Physical models of neural networks. World Scientific, Singapore · Zbl 0743.92004
[7] Hertz J, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley, Redwood City, California
[8] Kohonen T (1982a) Analysis of a simple self-organizing process. Biol Cybern 44:135–140 · Zbl 0495.93038 · doi:10.1007/BF00317973
[9] Kohonen T (1982b) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59–69 · Zbl 0466.92002 · doi:10.1007/BF00337288
[10] Kohonen T (1988) Self-organization and associative memory. Springer, New York Berlin Heidelberg · Zbl 0659.68100
[11] Kohonen T (1989) Speech recognition based on topology preserving neural maps. In: Aleksander I (ed) Neural computation. London, Kogan Page
[12] Kohonen T (1991) Self-organizing maps: optimization approaches. In: Kohonen T et al. (ed) Artificial neural networks, vol II. North Holland, Amsterdam, pp 981–990
[13] Lo ZP, Bavarian B (1991) On the rate of convergence in topology preserving neural networks. Biol Cybern 65:55–63 · Zbl 0731.92002 · doi:10.1007/BF00197290
[14] Luttrell SP (1989) Self-organization: a derivation from first principles of a class of learning algorithms. In: Proc. 3rd IEEE Int. Joint Conf. on Neural Networks, vol II. Washington, pp 495–498, IEEE Neural Networks Council
[15] Moon P, Spencer DE (1969) Partial differential equations. Heath, Lexington, Mass · Zbl 0223.35001
[16] Obermayer K, Ritter H, Schulten K (1990a) Large-scale simulations of self-organizing neural networks on parallel computers: Applications to biological modelling. Parallel Computer 14: 381–404 · doi:10.1016/0167-8191(90)90088-Q
[17] Obermayer K, Ritter H, Schulten K (1990b) A principle for the formation of the spatial structure of cortical feature maps. Proc Natl Acad Sci USA 87:8345–8349 · doi:10.1073/pnas.87.21.8345
[18] Obermayer K, Blasdel GG, Schulten K (1991) A neural network model for the formation of the spatial structure of retinotopic maps, orientationand ocular dominance columns. In: Kohonen T et al. (ed) Artificial neural networks, vol I. North Holland, Amsterdam, pp 505–511
[19] Ritter H (1988) Selbstorganisierende neuronale Karten. PhD thesis. Technische Universität München
[20] Ritter H, Kohonen T (1989) Self-organizing semantic maps. Biol Cybern 61:241–254 · doi:10.1007/BF00203171
[21] Ritter H, Schulten K (1986) On the stationary state of Kohonen’s self-organizing sensory mapping. Biol Cybern 54:99–106 · Zbl 0586.92004 · doi:10.1007/BF00320480
[22] Ritter H, Schulten K (1988) Convergence properties of Kohonen’s topology conserving mappings: Fluctuatuions, stability and dimension selection. Biol Cybern 60:59–71 · Zbl 0653.92026 · doi:10.1007/BF00205972
[23] Ritter H, Martinetz T, Schulten K (1989) Topology conserving maps for learning visuomotor-coordination. Neural Networks 2:159–168 · doi:10.1016/0893-6080(89)90001-4
[24] Robbins H, Munro S (1951) A stochastic approximation method. Ann Math Statist 22:400–407 · Zbl 0054.05901 · doi:10.1214/aoms/1177729586
[25] Schottes JC (1991) Unsupervised context learning in natural language processing. In: Proc 5th IEEE Int. Joint Conf. on Neural Networks, vol I. Washington, pp 107–112, IEEE Neural Networks Council
[26] Simic PD (1989) Statistical mechanics as the underlying theory of ’elastic’ and ’neural’ optimizations. Network 1:89–103 · Zbl 0850.82038 · doi:10.1088/0954-898X/1/1/007
[27] Tolat VV (1990) An analysis of Kohonen’s self-organizing maps using a system of energy functions. Biol Cybern 64:155–164 · Zbl 0711.92002 · doi:10.1007/BF02331345
[28] van Kämpen NG (1981) Stochastic process in physics and chemistry. North-Holland, Amsterdam
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.