Learning from heterogeneously distributed data sets using artificial neural networks and genetic algorithms

• Autorzy: Peteiro-Barral D. , Guijarro-Berdiñas B. , Pérez-Sánchez B.
• Języki publikacji: EN

Abstrakty

EN

It is a fact that traditional algorithms cannot look at a very large data set and plausibly find a good solution with reasonable requirements of computation (memory, time and communications). In this situation, distributed learning seems to be a promising line of research. It represents a natural manner for scaling up algorithms inasmuch as an increase of the amount of data can be compensated by an increase of the number of distributed locations in which the data is processed. Our contribution in this field is the algorithm Devonet, based on neural networks and genetic algorithms. It achieves fairly good performance but several limitations were reported in connection with its degradation in accuracy when working with heterogeneous data, i.e. the distribution of data is different among the locations. In this paper, we take into account this heterogeneity in order to propose several improvements of the algorithm, based on distributing the computation of the genetic algorithm. Results show a significative improvement of the performance of Devonet in terms of accuracy.

Słowa kluczowe

EN

artificial neural networks; genetic algorithm; Devonet algorithm

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2012
• Tom: Vol. 2, No. 1
• Strony: 5--20
• Opis fizyczny: Bibliogr. 29 poz., rys.

Twórcy

autor

Peteiro-Barral D.

• Department of Computer Science, University of A Coruña, Campus de Elviña s/n, 15071, A Coruña, Spain

autor

Guijarro-Berdiñas B.

• Department of Computer Science, University of A Coruña, Campus de Elviña s/n, 15071, A Coruña, Spain

autor

Pérez-Sánchez B.

• Department of Computer Science, University of A Coruña, Campus de Elviña s/n, 15071, A Coruña, Spain

Bibliografia

• [1] F. Provost and V. Kolluri. A survey of methods for scaling up inductive algorithms. Data mining and knowledge discovery, 3(2):131–169, 1999.
• [2] J. Catlett. Megainduction: machine learning on very large databases. PhD thesis, School of ComputerScience, University of Technology, Sydney, Australia, 1991.
• [3] L. Bottou and O. Bousquet. The tradeoffs of large scale learning. Advances in neural information processing systems, 20:161–168, 2008.
• [4] S. Sonnenburg, G. Ratsch, and K. Rieck. Large scale learning with string kernels. Large Scale Kernel Machines, pages 73–104, 2007.
• [5] C. Moretti, K. Steinhaeuser, D. Thain, and N.V. Chawla. Scaling up classifiers to cloud computers. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM), pages 472–481, 2008.
• [6] S. Krishnan, C. Bhattacharyya, and R. Hariharan. A randomized algorithm for large scale support vector learning. In Proceedings of Advances in Neural Information Processing Systems, pages 793–800, 2008.
• [7] R. Raina, A. Madhavan, and A.Y. Ng. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 873–880, 2009.
• [8] D. Sculley. Large scale learning to rank. In NIPS 2009 Workshop on Advances in Ranking, 2009.
• [9] L. Tang, S. Rajan, and V.K. Narayanan. Large scale multi-label classification via metalabeler. In Proceedings of the 18th international conference on World Wide Web, pages 211–220. ACM, 2009.
• [10] F.J. Huang and Y. LeCun. Large-scale learning with svm and convolutional for generic object categorization. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 284–291. IEEE, 2006.
• [11] S. Sonnenburg, G. Ratsch, S. Henschel, C. Widmer, J. Behr, A. Zien, F. de Bona, A. Binder, C. Gehl, and V. Franc. The SHOGUN Machine Learning Toolbox. Journal of Machine Learning Research, 11:1799–1802, 2010.
• [12] G. Tsoumakas. Distributed Data Mining. Database Technologies: Concepts, Methodologies, Tools, and Applications, pages 157–171, 2009.
• [13] P.K. Chan and S.J. Stolfo. Toward parallel and distributed learning by meta-learning. In AAAI workshop in Knowledge Discovery in Databases, pages 227–240, 1993.
• [14] W. Davies and P. Edwards. Dagger: A new approach to combining multiple models learned from disjoint subsets. Machine Learning, 2000:1–16, 2000.
• [15] A. Lazarevic and Z. Obradovic. The distributed boosting algorithm. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, page 316. ACM, 2001.
• [16] G. Tsoumakas and I. Vlahavas. Effective stacking of distributed classifiers. In ECAI 2002: 15th European Conference on Artificial Intelligence, July 21-26, 2002, Lyon France: including Prestigious Applications of Intelligent Systems (PAIS 2002): proceedings, page 340. Ios Pr Inc, 2002.
• [17] N. Chawla, L. Hall, K. Bowyer, T. Moore, and W. Kegelmeyer. Distributed pasting of small votes. Multiple Classifier Systems, pages 52–61, 2002.
• [18] B. Guijarro-Berdiñas, D. Mart´ınez-Rego, and S. Fernandez-Lorenzo. Privacy-Preserving Distributed Learning Based on Genetic Algorithms and Artificial Neural Networks. Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, pages 195–202, 2009.
• [19] E. Castillo, O. Fontenla-Romero, B. Guijarro-Berdiñas, and A. Alonso-Betanzos. A global optimum approach for one-layer neural networks. Neural Computation, 14(6):1429–1449, 2002.
• [20] O. Fontenla-Romero, B. Guijarro-Berdiñas, B. Pérez-Sánchez, and A. Alonso-Betanzos. A new convex objective function for the supervised learning of single-layer neural networks. Pattern Recognition, 43(5):1984–1992, 2010.
• [21] G. Carayannis, N. Kalouptsidis, and D.G. Manolakis. Fast recursive algorithms for a class of linear equations. IEEE Transactions on Acoustics, Speech, and Signal Processing, 30:227–239, 1982.
• [22] A. Bojańczyk. Complexity of solving linear systems in different models of computation. SIAM Journal on Numerical Analysis, 21(3):591–603, 1984.
• [23] S.M. Weiss and C.A. Kulikowski. Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems. Morgan Kaufmann, San Francisco, 1991.
• [24] A. Frank and A. Asuncion. UCI machine learning repository, 2010.
• [25] J. Kittler. Combining classifiers: A theoretical framework. Pattern Analysis & Applications, 1(1):18–27, 1998.
• [26] D.H. Wolpert. Stacked generalization. Neural networks, 5(2):241–259, 1992.
• [27] M. Hollander and D.A. Wolfe. Nonparametric statistical methods. 1999.
• [28] J.C. Hsu. Multiple comparisons: theory and methods. Chapman & Hall/CRC, 1996.
• [29] The MathWorks. MATLAB – The Language Of Technical Computing.http://www.mathworks.com/products/matlab/,2010. [Online; accessed 15-August-2010].