On Intra-Class Variance for Deep Learning of Classifiers

• Autorzy: Pilarczyk Rafał , Skarbek Władysław

Identyfikatory

DOI

10.2478/fcds-2019-0015

• Języki publikacji: EN

Abstrakty

EN

A novel technique for deep learning of image classifiers is presented. The learned CNN models higher offer better separation of deep features (also known as embedded vectors) measured by Euclidean proximity and also no deterioration of the classification results by class membership probability. The latter feature can be used for enhancing image classifiers having the classes at the model’s exploiting stage different from from classes during the training stage. While the Shannon information of SoftMax probability for target class is extended for mini-batch by the intra-class variance, the trained network itself is extended by the Hadamard layer with the parameters representing the class centers. Contrary to the existing solutions, this extra neural layer enables interfacing of the training algorithm to the standard stochastic gradient optimizers, e.g. AdaM algorithm. Moreover, this approach makes the computed centroids immediately adapting to the updating embedded vectors and finally getting the comparable accuracy in less epochs.

Słowa kluczowe

EN

deep learning of image classifier; convolutional neural network; Shanon information measure; intra-class variance; image embedding

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2019
• Tom: Vol. 44, No. 3
• Strony: 285--301
• Opis fizyczny: Bibliogr. 22 poz., rys., tab.

Twórcy

autor

Pilarczyk Rafał

• Warsaw University of Technology

autor

Skarbek Władysław

• Warsaw University of Technology

Bibliografia

• [1] Badrinarayanan V., Kendall A., and Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. arXiv:1511.00561 [cs], Nov. 2015. arXiv: 1511.00561.
• [2] Chen L.-C., Papandreou G., Kokkinos I., Murphy K., and Yuille A.L. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. arXiv:1606.00915 [cs], June 2016. arXiv: 1606.00915.
• [3] Fisher R.A. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7(2):179-188, Sept. 1936.
• [4] Hadsell R., Chopra S., and LeCun Y. Dimensionality reduction by learning an invariant mapping. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2, CVPR ’06, pages 1735-1742, Washington, DC, USA, 2006. IEEE Computer Society.
• [5] He K., Gkioxari G., DoMr P., and Girshick R. Mask R-CNN. arXiv:1703.06870 [cs], Mar. 2017. arXiv: 1703.06870.
• [6] He K., Zhang X., Ren S., and Sun J. Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs], Dec. 2015. arXiv: 1512.03385.
• [7] Jourabloo A. and Liu X. Large-Pose Face Alignment via CNN-Based Dense 3d Model Fitting. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4188-4196, Las Vegas, NV, USA, June 2016. IEEE.
• [8] Kingma D.P. and Ba J. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.
• [9] Kowalski M. and Naruniec J., (Warsaw University of Technology - personal communication).
• [10] Lecun Y. and Cortes C., The MNIST database of handwritten digits.
• [11] Ren S., He K., Girshick R., and Sun J.Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137-1149, June 2017.
• [12] Rosenblatt F., The perceptron-a perceiving and recognizing automaton. report 85-460-1. Technical report, Cornell Aeronautical Laboratory, 1957.
• [13] Rumelhart D.E., Hinton G.E., and Williams R.J., Learning representations by back-propagating errors. In Anderson J. A. and Rosenfeld E., editors, Neurocomputing: Foundations of Research, pages 696-699. MIT Press, Cambridge, MA, USA, 1988.
• [14] Schmidhuber J., Deep learning in neural networks: An overview. CoRR, abs/1404.7828, 2014.
• [15] Schroff F., Kalenichenko D., and Philbin J. FaceNet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815-823, Boston, MA, USA, June 2015. IEEE.
• [16] Simonyan K. and Zisserman A., Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1Ą09.1556 [cs], Sept. 2014. arXiv: 1409.1556.
• [17] Skarbek W., Symbolic tensor neural networks for digital media - from tensor processing via bnf graph rules to creams applications. Preprint stored in Cornell University Archive, abs/1809.06582, 2018.
• [18] Skarbek W., Kucharski K., and Bober M., Dual lda for face recognition. Fundam. Inform., 61:303-334, 07 2004.
• [19] Wen Y., Zhang K., Li Z., and Qiao Y. A Discriminative Feature Learning Approach for Deep Face Recognition. In Leibe B., Matas J., Sebe N., and Welling M., editors, Computer Vision - ECCV 2016, volume 9911, pages 499-515. Springer International Publishing, Cham, 2016.
• [20] Werbos P.J. Applications of advances in nonlinear sensitivity analysis. In Drenick R. F. and Kozin F., editors, System Modeling and Optimization, pages 762-770, Berlin, Heidelberg, 1982. Springer Berlin Heidelberg.
• [21] Xiao H., Rasul K., and Vollgraf R., Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXm:1708.077Ą7 [cs, stat], Aug. 2017. arXiv: 1708.07747.
• [22] Zhang K., Zhang Z., Li Z., and Qiao Y., Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23(10):1499-1503, Oct. 2016.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).