×

Artificial neural networks. A review from physical and mathematical points of view. (English) Zbl 0848.68085

Summary: This survey summarises recent mathematical results on the statistical mechanics of the Hopfield and Kac-Hopfield models.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
82C99 Time-dependent statistical mechanics (dynamic and nonequilibrium)
PDFBibTeX XMLCite
Full Text: Numdam EuDML

References:

[1] M. Aizenman , J. Lebowitz and D. Ruelle , Some rigorous results on the Sherrington-Kirkpatrick model , Com. Math. Phys. Vol. 112 , 1987 , pp. 3 - 20 . Article | MR 904135 · Zbl 1108.82312 · doi:10.1007/BF01217677
[2] D.J. Amit , Modeling Brain Function , Cambridge Univ. Press . Cambridge , 1988 . MR 1025121 | Zbl 0709.92001 · Zbl 0709.92001 · doi:10.1017/CBO9780511623257
[3] D.J. Amit , H. Gutfreund and H. Sompolinsky , Spin-glass models of neural networks , Phys. Rev. , Vol. A 32 , 1985 , pp. 1007 - 1018 . MR 797031
[4] D.J. Amit , H. Gutfreund and H. Sompolinsky , Statistical Mechanics of Neural Networks near Saturation , Ann. of Phys. , Vol. 173 , 1987 , pp. 30 - 67 .
[5] A. Bovier and V. Gayrard , Rigorous bounds on the storage capacity of the dilute Hopfield model , J. Stat. Phys. , Vol. 69 , 1992 , pp. 597 - 627 . MR 1193852 | Zbl 0900.82064 · Zbl 0900.82064 · doi:10.1007/BF01050427
[6] A. Bovier and V. Gayrard , Rigorous results on the thermodynamics of the dilute Hopfield model , J. Stat. Phys. , Vol. 69 , 1993 , pp. 597 - 627 . MR 1193852 · Zbl 1096.82517
[7] A. Bovier , V. Gayrard and P. Picco , Gibbs states of the Hopfield model in the regime of perfect memory , to appear in Prob. Theor. Rel. Fields , 1995 . MR 1305586 · Zbl 1081.82570
[8] A. Bovier , V. Gayrard and P. Picco , Large deviation principles for the Hopfield and the Kac-Hopfield model , to appear in Probability Theory and relatedfields , 1995 . Zbl 0826.60090 · Zbl 0826.60090 · doi:10.1007/BF01202783
[9] A. Bovier , V. Gayrard and P. Picco , Gibbs states for the Hopfield model with extensively many patterns , to appear in J.S.P. , 1995 . MR 1325589 | Zbl 01553930 · Zbl 1081.82570
[10] B. Cheng and D.M. Titterington , Neural Networks: A Review from a Statistical Perspective , Statistical Science , Vol. 9 , 1994 , pp. 2 - 54 . MR 1278678 | Zbl 0955.62589 · Zbl 0955.62589 · doi:10.1214/ss/1177010638
[11] F. Comets , Large deviation estimates for a conditional probability distribution. Applications to random Gibbs measures , Prob. Theor. Rel. Fields , Vol. 80 , 1989 , pp. 407 - 432 . MR 976534 | Zbl 0638.60037 · Zbl 0638.60037 · doi:10.1007/BF01794432
[12] M. Cassandro , E. Orlandi and E. Presutti , Interfaces and typical Gibbs configurations for one-dimensional Kac potentials , Prob. Theor. Rel. Fields , Vol. 96 , 1993 , pp. 57 - 96 . MR 1222365 | Zbl 0791.60096 · Zbl 0791.60096 · doi:10.1007/BF01195883
[13] R.S. Ellis , Entropy, large deviations, and statistical mechanics , Springer-Verlag , Berlin , 1985 . MR 793553 | Zbl 0566.60097 · Zbl 0566.60097
[14] P. Ferrari , S. Martinez and P. Picco , A lower bound for the memory capacity in the Hopfield-Potts model , J. Stat. Phys. , Vol. 66 , 1991 , pp. 1643 - 1651 . MR 1156420 | Zbl 0890.68110 · Zbl 0890.68110 · doi:10.1007/BF01054440
[15] L.A. Pastur and A.L. Figotin , Exactly soluble model of a spin glass , Sov. J. Low Temp. Phys. , Vol. 3 ( 6 ), 1977 , pp. 378 - 383 .
[16] L.A. Pastur and A.L. Figotin , On the theory of disordered spin systems , Theor. Math. Phys. , Vol. 35 , 1978 , pp. 403 - 414 . MR 503349 · Zbl 0433.35072
[17] L.A. Pastur and A.L. Figotin , Infinite range limit for a class of disordered spin systems , Theor. Math. Phys. , Vol. 51 , 1982 , pp. 564 - 569 . MR 674787
[18] V. Gayrard , The thermodynamics limit of the q-state Potts-Hopfield model with infinitely many patterns , J. Stat. Phys. , Vol. 68 , 1992 , pp. 977 - 1011 . MR 1180270 | Zbl 0900.92011 · Zbl 0900.92011 · doi:10.1007/BF01048882
[19] E. Gardner , The space of interactions in neural network models , J. Phys. A: Math. Gen. , Vol. 21 , 1988 , pp. 257 - 270 . MR 939730 · Zbl 1128.82302 · doi:10.1088/0305-4470/21/1/030
[20] E. Gardner and B. Derrida , Optimal storage properties of neural network models , J. Phys. A: Math. Gen. , Vol. 21 , 1988 , pp. 271 - 284 . MR 939731
[21] E. Goles , F. Fogelman-Soulie and D. Pellegrin , Decreasing energy functons as a tool for studying threshold networks , Discrete App. Math. , Vol. 12 , 1985 , pp. 261 - 277 . MR 813974 | Zbl 0585.94018 · Zbl 0585.94018 · doi:10.1016/0166-218X(85)90029-0
[22] E. Goles and S. Martinez , Neural and automata networks. Dynamical behavior and applications , Kluwer Ac. Pub. Dordrecht , Boston , London , 1990 . MR 1047470 · Zbl 1103.37302
[23] D. Grensing and K. Kühn , On classical spin-glass models , J. Phys. Paris , Vol. 48 , 1987 , pp. 713 - 721 . MR 896188
[24] M. Griniasty , M. Tsodyks and D.J. Amit , Neural Comp. , Vol. 5 , 1993 , pp. 1 - 17 .
[25] D.O. Hebb , The organisation of behavior , J. Willey & Son , New York , 1949 .
[26] J.J. Hopfield , Neural networks and physical systems with emergent collective computational abilities , Proc. Natl. Acad. Sci. USA , Vol. 79 , 1982 , pp. 2554 - 2558 . MR 652033 · Zbl 1369.92007
[27] I. Kanter and H. Sompolinsky , Associative recall of memory without errors , Phys. Rev. A , Vol. 35 , 1987 , pp. 380 - 392 .
[28] H. Koch , A free energy bound for the Hopfield model , J. Phys. A: Math Gen. , Vol. 26 , 1993 , pp. L353 - L355 . MR 1212003 | Zbl 0775.82008 · Zbl 0775.82008 · doi:10.1088/0305-4470/26/6/010
[29] T. Kohonen , Self-organisation and associative memory , Springer Verlag , Berlin , 1984 . MR 728946 | Zbl 0528.68062 · Zbl 0528.68062
[30] H. Koch and J. Piasko , Some rigorous results on the Hopfield neural network model , J. Stat. Phys. , Vol. 55 , 1989 , pp. 903 - 928 . MR 1002477 | Zbl 0716.92003 · Zbl 0716.92003 · doi:10.1007/BF01041071
[31] M. Kac , G. Uhlenbec and P.C. Hemmer , On the van der Waals theory of vapour-liquid equilibrium. I. Discussion of a one-dimensional model , J. Math. Phys. , Vol. 4 , 1963 , pp. 216 - 228 ; II. Discussion of the distribution functions , J. Math. Phys. , Vol. 4 , 1963 , pp. 229 - 247 ; III. Discussion of the critical region , J. Math. Phys. , Vol. 5 , 1964 , pp. 60 - 74 . MR 148416 | Zbl 0938.82517 · Zbl 0938.82517 · doi:10.1063/1.1703946
[32] J. Komlos and R. Paturi , Convergence results in an associative memory model , Neural Network. , Vol. 1 , 1988 , pp. 239 - 250 .
[33] J. Lebowitz and O. Penrose , Rigorous treatment of the Van der Waals Maxwell theory of the liquid-vapour transition , J. Math. Phys. , Vol. 7 , 1966 , pp. 98 - 113 . MR 187835 | Zbl 0938.82520 · Zbl 0938.82520 · doi:10.1063/1.1704821
[34] Y. Le Cun , B. Boser , J.S. Denker , D. Henderson , R.E. Howard , W. Hubbard and L.D. Jackel , Backpropagation applied to handwritten Zip code recognition , Neural Computation , Vol. 1 , 1989 , pp. 541 - 551 .
[35] Y. Le Cun , B. Boser , J.S. Denker , D. Henderson , R.E. Howard , W. Hubbard and L.D. Jackel , Handwritten digit recognition with a backpropagation network , Advances in Neural Information Processing Systems II (D. S. Touretzky, eds.), Morgan Kaufmann , San Mateo, CA , 1990 , pp. 396 - 404 .
[36] Y.C. Lee , G. Doolen , H.H. Chen , G.Z. Sun , T. Maxwell , H.Y. Lee and C.L. Giles , Machine learning using a higher order correlaton network , Physica D , Vol. 22 , 1987 , pp. 276 - 867 . MR 878166
[37] D. Loukianova , Capacité de mémoire dans le modèle de Hopfield , C. R. Acad. Sc. Paris , Serie 1 , Vol. 31 , 1994 , pp. 157 - 160 . MR 1260330 | Zbl 0794.92002 · Zbl 0794.92002
[38] R.J. McEliece , E.C. Posner , E.R. Rodemich and S.S. Venkatesh , The capacity of the Hopfield associative memory , IEEE Trans. Inform. Theory , Vol. 33 , 1987 , pp. 461 - 482 . MR 901677 | Zbl 0631.94016 · Zbl 0631.94016 · doi:10.1109/TIT.1987.1057328
[39] W.S. Mcculloch and W.A. Pitts , A logical calculus of the ideas immanent in nervous activity , Bul. Math. Biophysics , Vol. 5 , 1943 , pp. 115 - 133 . MR 10388 | Zbl 0063.03860 · Zbl 0063.03860 · doi:10.1007/BF02478259
[40] S. Martinez , Temuco summer school . To appear, 1992 .
[41] M. Minsky and S. Papert , Perceptrons , MIT Press , Cambridge MA , 1969 . · Zbl 0197.43702
[42] M. Mézard , J.P. Nadal and G. Toulouse , Solvable models of working memories , J. Phys. Paris , Vol. 47 , 1986 , pp. 1457 - 1462 . MR 930410
[43] M. Mézard , G. Parisi and M. Virasoro , Spin Glass Theory and Beyond , World Scientific , Singapore , 1987 . MR 1026102 | Zbl 0992.82500 · Zbl 0992.82500
[44] Y. Miyashita , Neuronal correlate of visual associative long-term memory in primate temporal cortex , Nature , Vol. 335 , 1988 , p. 817 .
[45] R. Monasson , Properties of neural networks storing spatially correlated patterns , J. Phys. A: Math. Gen. , Vol. 25 , 1992 , pp. 3701 - 3720 . MR 1172070 | Zbl 0791.68140 · Zbl 0791.68140 · doi:10.1088/0305-4470/25/13/019
[46] R. Monasson , Storage of spatially correlated patterns in auto-associative memories , J. Phys. Paris , Vol. 3 , 1993 , pp. 1141 - 1152 .
[47] C.M. Newman , Memory capacity in neural network models: rigorous lower bounds , Neural Networks , Vol. 1 , 1988 , pp. 223 - 238 .
[48] L. Pastur , M. Shcherbina and B. Tirozzi , The replica symmetric solution without the replica trick for the Hopfield model , J. Stat. Phys. , Vol. 74 , 1994 , pp. 1161 - 1183 . MR 1268789 | Zbl 0835.60092 · Zbl 0835.60092 · doi:10.1007/BF02188221
[49] L. Personnaz , I. Guyon and G. Dreyfus , Information storage and retrivial in Spin-glass like neural networks , J. Phys. Lett. , Vol. 49 , 1985 , pp. 359 - 365 .
[50] E. Scacciatelli and B. Tirozzi , Fluctuation of the free energy in the Hopfield model , J. Stat. Phys. , Vol. 67 , 1992 , pp. 981 - 1008 . MR 1170083 | Zbl 0925.82163 · Zbl 0925.82163 · doi:10.1007/BF01049007
[51] M. Shcherbina and B. Tirozzi , The free energy for a class of Hopfield models , J. Stat. Phys. , Vol. 72 , 1992 , pp. 113 - 125 . MR 1233028 · Zbl 1097.82551 · doi:10.1007/BF01048042
[52] W. Tarkowski and M. Lewenstein , Storage of sets of correlated data in neural network memories , J. Phys A: Math. Gen. , Vol. 26 , 1993 , pp. 2453 - 2469 . MR 1234407 | Zbl 0788.68118 · Zbl 0788.68118 · doi:10.1088/0305-4470/26/11/001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.