Projective nonnegative matrix factorization based on α-divergence

• Autorzy: Yang Z. , Oja E.
• Języki publikacji: EN

Abstrakty

EN

The well-known Nonnegative Matrix Factorization (NMF) method can be provided with more flexibility by generalizing the non-normalized Kullback-Leibler divergence to α- divergences. However, the resulting α-NMF method can only achieve mediocre sparsity for the factorizing matrices. We have earlier proposed a variant of NMF, called Projective NMF (PNMF) that has been shown to have superior sparsity over standard NMF. Here we propose to incorporate both merits of α-NMF and PNMF. Our α-PNMF method can produce a much sparser factorizing matrix, which is desired in many scenarios. Theoretically, we provide a rigorous convergence proof that the iterative updates of α-PNMF monotonically decrease the α-divergence between the input matrix and its approximate. Empirically, the advantages of α-PNMF are verified in two application scenarios: (1) it is able to learn highly sparse and localized part-based representations of facial images; (2) it outperforms α-NMF and PNMF for clustering in terms of higher purity and smaller entropy.

Słowa kluczowe

EN

Nonnegative Matrix Factorization; NMF; α-divergence; PNMF; α-NMF; α-PNMF

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2011
• Tom: Vol. 1, No. 1
• Strony: 7--16
• Opis fizyczny: Bibliogr. 23 poz., rys.

Twórcy

autor

Yang Z.

• Department of Information and Computer Science Aalto University School of Science and Technology P.O.Box 15400, FI-00076, Aalto, Finland

autor

Oja E.

• Department of Information and Computer Science Aalto University School of Science and Technology P.O.Box 15400, FI-00076, Aalto, Finland

Bibliografia

• [1] S. Amari. Differential-geometrical methods in statistics, volume 28 of Lecture Notes in Statistics. Springer-Verlag, New York, 1985.
• [2] J.-Ph. Brunet, P. Tamayo, T.R. Golub, and J.P. Mesirov. Metagenes and molecular pattern discovery using matrix factorization. Proceedings of the National Academy of Sciences, 101(12):4164–4169, 2004.
• [3] Seungjin Choi. Algorithms for orthogonal nonnegative matrix factorization. In Proceedings of IEEE International Joint Conference on Neural Networks, pages 1828–1832, 2008.
• [4] A. Cichocki, H. Lee, Y.-D. Kim, and S. Choi. Nonnegative matrix factorization with α-divergence. Pattern Recognition Letters, 29:1433–1440, 2008.
• [5] A. Cichocki, R. Zdunek, A.-H. Phan, and S. Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis. John Wiley, 2009.
• [6] I.S. Dhillon and S. Sra. Generalized nonnegative matrix approximations with bregman divergences. In Advances in Neural Information Processing Systems, volume 18, pages 283–290, 2006.
• [7] Chris Ding, Tao Li, and M.I. Jordan. Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1):45–55, 2010.
• [8] Chris Ding, Tao Li, Wei Peng, and Haesun Park. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 126–135, 2006.
• [9] K. Drakakis, S. Rickard, R. de Fr’ein, and A. Cichocki. Analysis of financial data using nonnegative matrix factorization. International Mathematical Forum, 3:1853–1870, 2008.
• [10] C. F’evotte, N. Bertin, and J.-L. Durrieu. Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation, 21(3):793–830, 2009.
• [11] D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788–791, 1999.
• [12] D. D. Lee and H. S. Seung. Algorithms for nonnegative matrix factorization. Advances in Neural Information Processing Systems, 13:556–562,2001.
• [13] W. Liu, N. Zheng, and X. Lu. Non-negative matrix factorization for visual coding. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), volume 3, pages 293–296, 2003.
• [14] A. Pascual-Montano, J.M. Carazo, Kieko Kochi, Dietrich Lehmann, and R. D. Pascual-Marqui. Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(3):403–415, 2006.
• [15] P.J. Phillips, H. Moon, S.A. Rizvi, and P.J. Rauss. The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence, 22:1090–1104, October 2000.
• [16] F. Samaria and A. Harter. Parameterisation of a stochastic model for human face identification. In Proceedings of 2nd IEEE Workshop on Applications of Computer Vision, pages 138–142, Sarasota FL, December 1994.
• [17] Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905, August 2000.
• [18] Zhirong Yang and Erkki Oja. Projective nonnegative matrix factorization with α-divergence. In Proceedings of 19th International Conference on Artificial Neural Networks (ICANN), pages 20–29, Limassol, Cyprus, 2009. Springer.
• [19] Zhirong Yang and Erkki Oja. Linear and nonlinear projective nonnegative matrix factorization. IEEE Transaction on Neural Networks, 2010. In press.
• [20] Zhirong Yang, Zhijian Yuan, and Jorma Laaksonen. Projective non-negative matrix factorization Journal on Pattern Recognition and Artificial Intelligence, 21(8):1353–1362, December 2007.
• [21] S.S. Young, P. Fogel, and D. Hawkins. Clustering scotch whiskies using non-negative matrix factorization. Joint Newsletter for the Section on Physical and Engineering Sciences and the Quality and Productivity Section of the American Statistical Association, 14(1):11–13, 2006.
• [22] Zhijian Yuan and Erkki Oja. Projective nonnegative matrix factorization for image compression and feature extraction. In Proc. of 14th Scandinavian Conference on Image Analysis (SCIA 2005), pages 333–342, Joensuu, Finland, June 2005.
• [23] Zhijian Yuan and Erkki Oja. A family of modified projective nonnegative matrix factorization algorithms. In Proceedings of the 9th International Symposium on Signal Processing and Its Applications (ISSPA), pages 1–4, 2007.