Generalized ordered linear regression with regularization

• Autorzy: Łęski J. , Henzel N.
• Języki publikacji: EN

Abstrakty

EN

Linear regression analysis has become a fundamental tool in experimental sciences. We propose a new method for parameter estimation in linear models. The 'Generalized Ordered Linear Regression with Regularization' (GOLRR) uses various loss functions (including the o-insensitive ones), ordered weighted averaging of the residuals, and regularization. The algorithm consists in solving a sequence of weighted quadratic minimization problems where the weights used for the next iteration depend not only on the values but also on the order of the model residuals obtained for the current iteration. Such regression problem may be transformed into the iterative reweighted least squares scenario. The conjugate gradient algorithm is used to minimize the proposed criterion function. Finally, numerical examples are given to demonstrate the validity of the method proposed.

Słowa kluczowe

EN

linear regression; IRLS; OWA; conjugate gradient optimization; robust methods

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2012
• Tom: Vol. 60, nr 3
• Strony: 481--489
• Opis fizyczny: Bibliogr. 20 poz., rys., tab.

Twórcy

autor

Łęski J.

autor

Henzel N.

• Institute of Electronics, Silesian University of Technology, 16 Akademicka St., 44-100 Gliwice, Poland

Bibliografia

• [1] A.M. Legendre, Nouvelles Methodes Pour la Determinationdes Orbites des Cometes, Didot, Paris, 1805.
• [2] K.F. Gauss, Theory of the Motion of the Heavenly Bodies MovingAbout the Sun in Conic Sections: a translation of Gauss’sTheoria Motus, Little, Brown and Company, Boston, 1857.
• [3] R. Deutsch, Estimation Theory, Prentice-Hall, Englewood Cliffs, 1965.
• [4] J.M. Łęski, “ǫ-insensitive fuzzy c-regression models: Introduction to ǫ-insensitive fuzzy modeling”, IEEE Trans. Syst., Manand Cybern. - Part B: Cybern. 34 (1), 4-15 (2004).
• [5] J.M. Łęski, “TSK-fuzzy modeling based on ǫ-insensitive learning”, IEEE Trans. Fuzzy Syst. 13 (2), 181-193 (2005).
• [6] J.M. Łęski, “Iteratively reweighted least squares classifier and its ℓ2- and ℓ1-regularized kernel versions”, Bull. Pol. Ac.: Tech. 58 (1), 171-182 (2010).
• [7] V. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 1995.
• [8] J. Dziekan, W. Matczak, and J. Korbicz, “Active fault-tolerant control design for Takagi-Sugeno fuzzy systems”, Bull. Pol.Ac.: Tech. 59 (1), 93-102 (2011).
• [9] M. Kaminski and T. Orlowska-Kowalska, “Optimisation of neural state variables estimators of two-mass drive system using the Bayesian regularization method”, Bull. Pol. Ac.: Tech. 59 (1), 33-38 (2011).
• [10] P.J. Huber, “Robust estimation of location parameter”, Ann.Math. Stat. 35 (1), 73-101 (1964).
• [11] P.J. Huber, Robust Statistics, Wiley, New York, 1981.
• [12] P.J. Rousseeuw and A.M. Leroy, Robust Regression and OutliersDetection, John Wiley, New York, 1987.
• [13] V. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
• [14] R.R. Yager, “On ordered weighted averaging aggregation operators in multicriteria decision making”, IEEE Trans. Syst.,Man Cybern. 18(1), 183-190 (1988).
• [15] J. Fodor and M. Roubens, Fuzzy Preference Modelling andMulticriteria Decision Support, Kluwer, Dordrecht, 1994.
• [16] J.M. Łęski and N. Henzel, “ECG baseline wander and powerline interference reduction using nonlinear filter bank”, SignalProcessing 85 (2), 781-793 (2005).
• [17] R.R. Yager, “OWA operators in regression problems”, IEEETrans. Fuzzy Systems 18 (1), 106-113 (2010).
• [18] E.J. Schlossmacher, “An iterative technique for absolute deviations curve fitting”, J. Amer. Statist. Assoc. 68 (344), 857-859 (1973).
• [19] P. Holland and R. Welsch, “Robust regression using iteratively reweighted least-squares”, Commun. Stat. Theoret. Meth. 6 (9), 813-827 (1977).
• [20] D.G. Luenberger, Linear and Nonlinear Programming, Kluwer Acad. Press, Boston, 2003.