The recognition of partially occluded objects with support vector machines, convolutional neural networks and deep belief networks

• Autorzy: Chu J. L. , Krzyżak A.

Identyfikatory

DOI

10.2478/jaiscr-2014-0021

• Języki publikacji: EN

Abstrakty

EN

Biologically inspired artificial neural networks have been widely used for machine learning tasks such as object recognition. Deep architectures, such as the Convolutional Neural Network, and the Deep Belief Network have recently been implemented successfully for object recognition tasks. We conduct experiments to test the hypothesis that certain primarily generative models such as the Deep Belief Network should perform better on the occluded object recognition task than purely discriminative models such as Convolutional Neural Networks and Support Vector Machines. When the generative models are run in a partially discriminative manner, the data does not support the hypothesis. It is also found that the implementation of Gaussian visible units in a Deep Belief Network trained on occluded image data allows it to also learn to effectively classify non-occluded images.

Słowa kluczowe

EN

neural networks; belief networks; convolutional neural networks; artificial neural networks; Deep Belief Network; generative model

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2014
• Tom: Vol. 4, No. 1
• Strony: 5--19
• Opis fizyczny: Bibliogr. 25 poz., rys.

Twórcy

autor

Chu J. L.

• Department of Computer Science, Concordia University, Montreal, Quebec, Canada

autor

Krzyżak A.

• Department of Computer Science, Concordia University, Montreal, Quebec, Canada

Bibliografia

• [1] D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science, 9:147–169, 1985.
• [2] Y. Bengio. Learning deep architectures for ai. Foundations and Trends in Machine Learning, 2(1):1–127, 2009.
• [3] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27, 2011. Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
• [4] Joseph Lin Chu and Adam Krzy˙zak. Application of support vector machines, convolutional neural networks and deep belief networks to recognition of partially occluded objects. In L. Rutkowski, editor, The 13th International Conference on Artificial Intelligence and Soft Computing ICAISC 2014, Lecture Notes on Artifical Intelligece (LNAI), volume 8467, pages 34–46. Springer International Publishing Switzerland, 2014.
• [5] R. Collobert and S. Bengio. Links between perceptrons, mlps and svms. Proceedings of the 21st International Conference on Machine Learning, page 23, 2004.
• [6] C. Cortes and V. N. Vapnik. Support-vector networks. Machine Learning, 20:273–297, 1995.
• [7] K. Fukushima. Neocognitron for handwritten digit recognition. Neurocomputing, 51:161–180, 2003.
• [8] Kunihiko Fukushima and Sei Miyake. Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition, 15(6):455–469, 1982.
• [9] G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771–1800, 2002.
• [10] G. E. Hinton. A practical guide to training restricted boltzmann machines. Momentum, 9(1):599–619, 2010.
• [11] G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527–1554, 2006.
• [12] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313:504–507, 2006.
• [13] J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the USA, 79(8):2554–2558, 1982.
• [14] F. J. Huang and Y. LeCun. Large-scale learning with svm and convolutional nets for generic object categorization. Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1:284–291, 2006.
• [15] D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in a cat’s visual cortex. Journal of Physiology (London), 160:106–154, 1962.
• [16] Y. LeCun, L. Bottou, Y. Bengio, and Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
• [17] Y. LeCun, F.J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2:97–104, 2004.
• [18] Aleix M. Mart´ınez. Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6):748–763, 2002.
• [19] V. Nair and G. E. Hinton. 3d object recognition with deep belief nets. Advances in Neural Information Processing Systems (NIPS), pages 1339–1347, 2009.
• [20] M. Ranzato, J. Susskind, V. Mnih, and G. Hinton. On deep generative models with applications to recognition. 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2857–2864, 2011.
• [21] M. A. Ranzato, F. J. Huang, Y. L. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–8, 2007.
• [22] P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In David E. Rumelhart and James L. McLelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1, chapter 6, pages 194–281. MIT Press, 1986.
• [23] P.W. M. Tsang and P. C. Yuen. Recognition of partially occluded objects. IEEE Transactions on Systems, Man and Cybernetics, 23(1):228–236, 1993.
• [24] John Winn and Jamie Shotton. The layout consistent random field for recognizing and segmenting partially occluded objects. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 1:37–44, 2006.
• [25] LaurenzWiskott and Christoph Von Der Malsburg. A neural system for the recognition of partially occluded objects in cluttered scenes: A pilot study. International Journal of Pattern Recognition and Artificial Intelligence, 7(4):935–948, 1993.