×

Least squares for practitioners. (English) Zbl 1195.62015

Summary: In experimental sciences and engineering, least squares are ubiquitous in analysis and digital data processing applications. Minimizing sums of squares of some quantities can be interpreted in very different ways and confusion can arise in practice, especially concerning the optimality and reliability of the results. Interpretations of least squares in terms of norms and likelihoods need to be considered to provide guidelines for general users. Assuming minimal prerequisites, the following expository discussion is intended to elaborate on some of the mathematical characteristics of the least-squares methodology and some closely related questions in the analysis of the results, model identification, and reliability for practical applications. Examples of simple applications are included to illustrate some of the advantages, disadvantages, and limitations of least squares in practice. Concluding remarks summarize the situation and provide some indications of practical areas of current research and development.

MSC:

62F10 Point estimation
62P30 Applications of statistics in engineering and industry; control charts
62P99 Applications of statistics

Software:

Mathematica
PDFBibTeX XMLCite
Full Text: DOI EuDML

References:

[1] T. Hall, Carl Friedrich Gauss: A Biography, Translated from the Swedish by Albert Froderberg, The MIT Press, Cambridge, Mass, USA, 1970. · Zbl 0229.01015
[2] E. T. Jaynes, Probability Theory, Cambridge University Press, Cambridge, UK, 2003. · Zbl 1045.62001
[3] A. Egger, “Maximum likelihood and best approximations,” The Rocky Mountain Journal of Mathematics, vol. 20, no. 1, pp. 117-122, 1990. · Zbl 0699.62003 · doi:10.1216/rmjm/1181073164
[4] A. J. Pope, “Two approaches to nonlinear least squares adjustments,” The Canadian Surveyor, vol. 28, no. 5, pp. 663-669, 1974.
[5] G. Golub and V. Pereyra, “Separable nonlinear least squares: the variable projection method and its applications,” Inverse Problems, vol. 19, no. 2, pp. R1-R26, 2003. · Zbl 1022.65014 · doi:10.1088/0266-5611/19/2/201
[6] D. Pollard and P. Radchenko, “Nonlinear least-squares estimation,” Journal of Multivariate Analysis, vol. 97, no. 2, pp. 548-562, 2006. · Zbl 1085.62027 · doi:10.1016/j.jmva.2005.04.002
[7] H. D. Sherali and W. P. Adams, A Reformulation-Linearization Technique for Solving Discrete and Continuous Nonconvex Problems, vol. 31 of Nonconvex Optimization and Its Applications, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1999. · Zbl 0926.90078
[8] A. E. Hoerl and R. W. Kennard, “Ridge regression: biased estimation for nonorthogonal problems,” Technometrics, vol. 12, pp. 55-67, 1970. · Zbl 0202.17205 · doi:10.2307/1267351
[9] A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, John Wiley & Sons, New York, NY, USA, 1977. · Zbl 0354.65028
[10] Y. C. Eldar and A. V. Oppenheim, “Covariance shaping least-squares estimation,” IEEE Transactions on Signal Processing, vol. 51, no. 3, pp. 686-697, 2003. · doi:10.1109/TSP.2002.808125
[11] J. A. R. Blais, “Generalized covariance functions and their applications in estimation,” Manuscripta Geodaetica, vol. 9, no. 4, pp. 307-312, 1984. · Zbl 0549.62056
[12] C. A. Micchelli, “Interpolation of scattered data: distance matrices and conditionally positive definite functions,” Constructive Approximation, vol. 2, no. 1, pp. 11-22, 1986. · Zbl 0625.41005 · doi:10.1007/BF01893414
[13] J. A. R. Blais, Estimation and Spectral Analysis, University of Calgary Press, Calgary, AB, Canada, 1988, http://www.netlibrary.com/. · Zbl 0747.62098
[14] Å. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, Pa, USA, 1996. · Zbl 0847.65023
[15] V. C. Klema and A. J. Laub, “The singular value decomposition: its computation and some applications,” IEEE Transactions on Automatic Control, vol. 25, no. 2, pp. 164-176, 1980. · Zbl 0433.93018 · doi:10.1109/TAC.1980.1102314
[16] Gerald R. North, “Empirical orthogonal functions and normal modes,” Journal of the Atmospheric Sciences, vol. 41, no. 5, pp. 879-887, 1984.
[17] R. W. Preisendorfer, Principle Components and the Motions of Simple Dynamical Systems, Scripps Institution of Oceanography, 1979, Ref. Ser. 70-11.
[18] K.-Y. Kim and Q. Wu, “A comparison study of EOF techniques: analysis of nonstationary data with periodic statistics,” Journal of Climate, vol. 12, no. 1, pp. 185-199, 1999.
[19] Wolfram: Mathematica 7. Wolfram Research Inc., Champaign, Ill, USA, 2008.
[20] G. Eshel, “Geosci236: Empirical Orthogonal Functions,” Technical Note, Department of the Geophysical Sciences, University of Chicago, 2005.
[21] D. Treebushny and H. Madsen, “A new reduced rank square root Kalman filter for data assimilation in mathematical models,” in Proceedings of the International Conference in Computational Science (ICCS ’03), P. M. A. Sloot, D. Abramson, A. V. Bogdanov, J. C. Dongarra, A. Y. Zomaya, and Y. E. Gorbachev, Eds., vol. 2657 of Lecture Notes in Computer Science, pp. 482-491, 2003. · Zbl 1033.93500
[22] H. Akaike, “A new look at the statistical model identification,” IEEE Transactions on Automatic Control, vol. 19, pp. 716-723, 1974. · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705
[23] H. Akaike, “Information theory and an extension of the maximum likelihood principle,” in Second International Symposium on Information Theory (Tsahkadsor, 1971), pp. 267-281, Akadémiai Kiadó, Budapest, Hungary, 1973. · Zbl 0283.62006
[24] J. A. R. Blais, “On some model identification strategies using information theory,” Manuscripta Geodaetica, vol. 16, no. 5, pp. 326-332, 1991.
[25] J. A. R. Blais, “Reliability considerations in geospatial information systems,” Geomatica, vol. 56, no. 4, pp. 341-350, 2002.
[26] P. Meissl, “A Priori Prediction of Roundoff Error Accumulation in the Solution of a Super Large Geodetic Normal Equation System,” NOAA Professional Paper 12, National Geodetic Information Branch, National Oceanic and Atmospheric Administration (NOAA), Rockville, Md, USA, 1980.
[27] C. R. Schwartz, Ed., North American Datum of 1983. NOAA Professional Paper NOS 2, National Oceanic and Atmospheric Administration (NOAA), US Department of Commerce, 1989.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.