×

Relevance regression learning with support vector machines. (English) Zbl 1205.68279

Summary: We propose a variant of two SVM regression algorithms expressly tailored in order to exploit additional information summarizing the relevance of each data item, as a measure of its relative importance w.r.t. the remaining examples. These variants, enclosing the original formulations when all data items have the same relevance, are preliminary tested on synthetic and real-world data sets. The obtained results outperform standard SVM approaches to regression if evaluated in light of the above mentioned additional information about data quality.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Cortes, C.; Vapnik, V., Support-vector networks, Machine Learning, 20, 121-167 (1995)
[2] Vapnik, V., The Nature of Statistical Learning Theory (1995), Springer: Springer New York · Zbl 0833.62008
[3] Drucker, H.; Burges, C. J.C.; Kaufman, L.; Smola, A.; Vapnik, V., Support vector regression machines, (Mozer, M. C.; Jordan, M. I.; Petsche, T., Advances in Neural Information Processing Systems, vol. 9 (1997), MIT Press: MIT Press Cambridge, MA), 155-161
[4] Smola, A.; Scölkopf, B., A tutorial on support vector regression, Statistics and Computing, 14, 199-222 (2004)
[5] Feltham, G., The value of information, Accounting Review, 43, 4, 684-696 (1968)
[6] Burr, I. W., Engineering Statistics and Quality Control (1953), McGraw-Hill · Zbl 0052.14802
[7] Garvin, D. A., Quality on the line, Harward Business Review, 61, 5, 65-75 (1983)
[8] Te’eni, D., Behavioral aspects of data production and teir impact on data quality, Journal of Database Management, 4, 2, 30-38 (1993)
[9] Carroll, R.; Ruppert, D., Transformation and Weighting in Regression (1988), Chapman and Hall: Chapman and Hall New York · Zbl 0666.62062
[10] Ryan, T., Modern Regression Methods (1997), Wiley: Wiley New York · Zbl 0885.62074
[11] F. Dufrenois, J. Colliez, D. Hamad, Weighted support vector regression for robust single model estimation: application to motion segmentation in image sequences, in: Neural Networks, 2007, IJCNN 2007, International Joint Conference on, 2007, pp. 586-591.; F. Dufrenois, J. Colliez, D. Hamad, Weighted support vector regression for robust single model estimation: application to motion segmentation in image sequences, in: Neural Networks, 2007, IJCNN 2007, International Joint Conference on, 2007, pp. 586-591.
[12] Cawley, G. C.; Talbot, N. L.C.; Foxall, R. J.; Dorling, S. R.; Mandic, D. P., Heteroscedastic kernel ridge regression, Neurocomputing, 57, 105-124 (2004)
[13] Jordaan, E. M.; Smits, G. F., Robust outlier detection using SVM regression, (Neural Networks, 2004. Neural Networks, 2004, Proceedings. 2004 IEEE International Joint Conference, vol. 3 (2004)), 2017-2022
[14] Mangasarian, O.; Musicant, D., Robust linear and support vector regression, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 9, 950-955 (2000)
[15] Blitzer, J.; Crammer, K.; Kulesza, A.; Pereira, F.; Wortman, F., Learning bounds for domain adaptation, (Advances in Neural Information Processing Systems, vol. 20 (2008), MIT Press: MIT Press Cambridge, MA), 129-136
[16] Smola, A. J.; Murata, N.; Scölkopf, B.; Müller, K. R., Asymptotically optimal choice of \(\epsilon \)-loss for support vector machines, (Proceedings of the 8th International Conference on Artificial Neural Networks, Perspectives in Neural Computing (1998), Springer-Verlag), 105-110
[17] Chalimourda, A.; Schölkopf, B.; Smola, A. J., Choosing \(\nu\) in support vector regression with different noise models-theory and experiments, (IEEE-INNS-ENNS International Joint Conference on Neural Networks (2000), IEEE Computer Society), 5199
[18] Malchiodi, D.; Bassis, S.; Valerio, L., Discovering regression data quality through clustering methods, (New Directions in Neural Networks. New Directions in Neural Networks, 18th Italian Workshop on Neural Networks: WIRN 2008, 22-24 May 2008 (2009), IOS Press: IOS Press Vietri sul Mare), 76-85
[19] Apolloni, B.; Malchiodi, D.; Natali, L., A modified SVM classification algorithm for data of variable quality, (Knowledge-Based Intelligent Information and Engineering Systems 2007, Part III (2007), Springer-Verlag: Springer-Verlag Berlin, Heidelberg), 131-139
[20] Crammer, K.; Kearns, M.; Wortman, J., Learning from data of variable quality, (Weiss, Y.; Schölkopf, B.; Platt, J., Advances in Neural Information Processing Systems, vol. 18 (2006), MIT press: MIT press Cambridge, MA), 219-226
[21] M.E. Tipping, Sparse bayesian learning and the relevance vector machine, Journal of Machine Learning Research. 1 (201) 211-244.; M.E. Tipping, Sparse bayesian learning and the relevance vector machine, Journal of Machine Learning Research. 1 (201) 211-244. · Zbl 0997.68109
[22] MacKay, J. C., Bayesian methods for backpropagation networks, (Domany, E.; van Hemmen, J. L.; Schulten, K., Models of Neural Networks III (1994), Springer), 211-254, ch. 6
[23] Scölkopf, B.; Smola, A., Learning with Kernels (2002), MIT Press: MIT Press Cambridge, MA
[24] G. Schmidt, R. Mattern, F. Schueler, Biomechanical investigation to determine physical and traumatological differentiating criteria for the maximum load capacity of head and vertebral column with and without protective helmet under the effects of impact, in: EEC Research Program on Biomechanics of Impact, Final Report., Tech. Rep., University of Heidelberg, 1981.; G. Schmidt, R. Mattern, F. Schueler, Biomechanical investigation to determine physical and traumatological differentiating criteria for the maximum load capacity of head and vertebral column with and without protective helmet under the effects of impact, in: EEC Research Program on Biomechanics of Impact, Final Report., Tech. Rep., University of Heidelberg, 1981.
[25] L.H. Cox, M.M. Johnson, K. Kafadar, Exposition of statistical graphics technology, in: ASA Proc. Stat. Comp. Section, 1982, pp. 55-56.; L.H. Cox, M.M. Johnson, K. Kafadar, Exposition of statistical graphics technology, in: ASA Proc. Stat. Comp. Section, 1982, pp. 55-56.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.