\input zb-basic
\input zb-ioport
\iteman{io-port 00778144}
\itemau{Dixon, L.C.W.}
\itemti{Neural networks and unconstrained optimization.}
\itemso{Spedicato, Emilio (ed.), Algorithms for continuous optimization: the state of the art. Proceedings of the NATO Advanced Study Institute, Il Ciocco, Barga, Italy, September 5-18, 1993. Dordrecht: Kluwer Academic Publishers. NATO ASI Ser., Ser. C, Math. Phys. Sci. 434, 513-530 (1994).}
\itemab
Summary: When performing the unconstrained optimization of a complicated industrial problem, the main computational time is usually spent in the calculation of the objective function and its derivatives. The calculation of the objective function is usually performed sequentially, so if a parallel processing machine is to be used, then either a number of function evaluations must be calculated in parallel or the sequential calculation of the objective function replaced by a parallel calculation. The first approach has led to many codes, some of which are efficient. The second approach is very efficient when the objective function has the appropriate, partially separable structure. In this paper a third possibility is introduced where a neural network is used to represent the objective function calculation. A neural network is essential a parallel processing computer. It is well-known that a neural network with a single hidden layer can reproduce accurately any continuous function on a compact set. The task of calculating the sequential objective function during the optimization is then replaced by a learning task of assigning the correct parameters to the neural network. This learning task is itself a least squares, unconstrained optimization problem. In the neural network field this is often solved by back propagation. For the neural network structure it is possible to apply reverse automatic differentiation effectively to obtain the gradient and Hessian of the learning function and results then show that the truncated Newton method is literally thousands of times faster than back propagation. This study compares the performance of radial basis function networks with the usual sigmoid networks, but concludes by showing that nets based on the ``sinc'' function are even better for approximating continuous functions. While a previously published study was restricted to data points on regular grids, this is extended in this paper to more random distributions.
\itemrv{~}
\itemcc{}
\itemut{neural network; objective function calculation; parallel processing; learning}
\itemli{}
\end