×

FPGA implementation of a wavelet neural network with particle swarm optimization learning. (English) Zbl 1144.68342

Summary: This paper introduces implementation of a wavelet neural network (WNN) with learning ability on field programmable gate array (FPGA). A learning algorithm using gradient descent method is not easy to implement in an electronic circuit and has local minimum. A more suitable method is the particle swarm optimization (PSO) that is a population-based optimization algorithm. The PSO is similar to the GA, but it has no evolution operators such as crossover and mutation. In the approximation of a nonlinear activation function, we use a Taylor series and a look-up table (LUT) to achieve a more accurate approximation. The results of the two experiments demonstrate the successful hardware implementation of the wavelet neural networks with the PSO algorithm using FPGA. From the results of the experiment, it can be seen that the performance of the PSO is better than that of the simultaneous perturbation algorithm at sufficient particle sizes.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
65T60 Numerical methods for wavelets
82C32 Neural nets applied to problems in time-dependent statistical mechanics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] G. Mealing, M. Bani-Yaghoub, R. Tremblay, R. Monette, J. Mielke, R. Voicu, C. Py, R. Barjovanu, K. Faid, Application of polymer microstructures with controlled surface chemistries as a platform for creating and interfacing with synthetic neural networks, in: Proceedings. IEEE International Joint Conference on Neural Networks, vol. 5, 31 July-4 Aug. 2005, pp. 3116-3120; G. Mealing, M. Bani-Yaghoub, R. Tremblay, R. Monette, J. Mielke, R. Voicu, C. Py, R. Barjovanu, K. Faid, Application of polymer microstructures with controlled surface chemistries as a platform for creating and interfacing with synthetic neural networks, in: Proceedings. IEEE International Joint Conference on Neural Networks, vol. 5, 31 July-4 Aug. 2005, pp. 3116-3120
[2] Bhatikar, S. R.; Mahajan, R. L., Artificial neural-network-based diagnosis of CVD barrel reactor, IEEE Transactions on Semiconductor Manufacturing, 15, 1, 71-78 (2002)
[3] Choy, M. C.; Srinivasan, D.; Cheu, R. L., Neural networks for continuous online learning and control, IEEE Transactions on Neural Networks, 17, 6, 1511-1531 (2006)
[4] Markou, M.; Singh, S., A neural network-based novelty detector for image sequence analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 10, 1664-1677 (2006)
[5] G.J. Gibson, S. Siu, C.F.N. Cowan, Application of multilayer perceptrons as adaptive channel equalizers, in: Proc. IEEE Conf. Acoust. Speech Signal Process., 1989, pp. 1183-1186; G.J. Gibson, S. Siu, C.F.N. Cowan, Application of multilayer perceptrons as adaptive channel equalizers, in: Proc. IEEE Conf. Acoust. Speech Signal Process., 1989, pp. 1183-1186
[6] Rumelbart, D. E.; Hinton, G. E.; Williams, R. J., Learning internal representation by error propagation, (Rumelhart, D. E.; McClelland, J. L., Parallel/Distributed Processing Exploration in yth Microstructure of Cognition (1986), MIT Press: MIT Press Cambridge, MA), 318-362
[7] Hornik, K., Multilayer feedforward networks are universal approximators, Neural Networks, 2, 359-366 (1989) · Zbl 1383.92015
[8] Ikonomopoulos, A.; Endou, A., Wavelet decomposition and radial basis function networks for system monitoring, IEEE Transactions on Nuclear Science, 45, 5, 2293-2301 (1998)
[9] Ho, D. W.C.; Zhang, P. A.; Xu, J., Fuzzy wavelet networks for function learning, IEEE Transactions on Fuzzy Systems, 9, 1, 200-211 (2001)
[10] Zhang, Q., Using wavelet networks in nonparametric estimation, IEEE Transactions on Neural Networks, 8, 227-236 (1998)
[11] Lin, F. J.; Wai, R. J.; Chen, M. P., Wavelet neural network control for linear ultrasonic motor drive via adaptive sliding-mode technique, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 50, 6, 686-698 (2003)
[12] Huang, Y. C.; Huang, C. M., Evolving wavelet networks for power transformer condition monitoring, IEEE Transactions on Power Delivery, 17, 2, 412-416 (2002)
[13] Lin, F. J.; Lin, C. H.; Shen, P. H., Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive, IEEE Transactions on Fuzzy Systems, 9, 5, 751-759 (2001)
[14] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proc. IEEE Int. Conf. on Neural Networks, 1995, pp. 1942-1948; J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proc. IEEE Int. Conf. on Neural Networks, 1995, pp. 1942-1948
[15] Gaing, Z. L., A particle swarm optimization approach for optimum design of PID controller in AVR system, IEEE Transactions on Energy Conversion, 19, 2, 384-391 (2004)
[16] Yoshida, H.; Kawata, K.; Fukuyama, Y.; Takayama, S.; Nakanishi, Y., A particle swarm optimization for reactive power and voltage control considering voltage security assessment, IEEE Transactions on Power Systems, 15, 4, 1232-1239 (2000)
[17] Abido, M. A., Optimal design of power-system stabilizers using particle swarm optimization, IEEE Transactions on Energy Conversion, 17, 3, 406-413 (2002)
[18] Juang, C. F., A hybrid of genetic algorithm and particle swarm optimization for recurrent network design, IEEE Transactions on Systems, Man and Cybernetics, Part B, 34, 2, 997-1006 (2004)
[19] R. Mendes, P. Cortez, M. Rocha, J. Neves, Particle swarms for feedforward neural network training, in: The 2002 International Joint Conference on Neural Networks, 2002, pp. 1895-1899; R. Mendes, P. Cortez, M. Rocha, J. Neves, Particle swarms for feedforward neural network training, in: The 2002 International Joint Conference on Neural Networks, 2002, pp. 1895-1899
[20] Botros, N. M.; Abdul-Aziz, M., Hardware implementation of an artificial neural network using field programmable gate arrays (FPGA’s), IEEE Transactions on Industrial Electronics, 41, 6, 665-667 (1994)
[21] Blake, J. J.; Maguire, L. P.; McGinnity, T. M.; Roche, B.; McDaid, L. J., The implementation of fuzzy systems, neural networks and fuzzy neural networks using FPGAs, Information Sciences, 112, 1-4, 151-168 (1998)
[22] H. Faiedh, Z. Gafsi, K. Torki, K. Besbes, Digital hardware implementation of a neural network used for classification, in: The 16th International Conference on Microelectronics, Dec. 2004, pp. 551-554; H. Faiedh, Z. Gafsi, K. Torki, K. Besbes, Digital hardware implementation of a neural network used for classification, in: The 16th International Conference on Microelectronics, Dec. 2004, pp. 551-554
[23] Yen, C. T.; Wang, W. D.; Lin, Y. T., FPGA realization of a neural-network-based nonlinear channel equalizer, IEEE Transactions on Industrial Electronics, 51, 2, 472-479 (2004)
[24] Maeda, Y.; Tada, T., FPGA implementation of a pulse density neural network with learning ability using simultaneous perturbation, IEEE Transaction on Neural Networks, 14, 3, 688-695 (2003)
[25] Hikawa, H., A digital hardware pulse-mode neuron with piecewise linear activation function, IEEE Transactions on Neural Networks, 14, 5, 1028-1037 (2003)
[26] C.J. Lin, C.C. Shih, P.Y. Chen, A nonlinear time-varying channel equalizer using self-organizing wavelet neural networks, in: IEEE Int’l Joint Conf. on Neural Networks, Budapest, 25-29 July 2004, pp. 2089-2094; C.J. Lin, C.C. Shih, P.Y. Chen, A nonlinear time-varying channel equalizer using self-organizing wavelet neural networks, in: IEEE Int’l Joint Conf. on Neural Networks, Budapest, 25-29 July 2004, pp. 2089-2094
[27] C.J. Lin, Ch.Y. Lee, C.H. Chen, A self-adaptive quantum radial basis function network for classification applications, in: International Joint Conference on Neural Networks, Budapest, July 2004; C.J. Lin, Ch.Y. Lee, C.H. Chen, A self-adaptive quantum radial basis function network for classification applications, in: International Joint Conference on Neural Networks, Budapest, July 2004
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.