CMIM-2: An Enhanced Conditional Mutual Information Maximization Criterion for Feature Selection

• Autorzy: Vergara J. R. , Estévez P. A.
• Języki publikacji: EN

Abstrakty

EN

A new greedy feature selection criterion is proposed as an enhancement of the conditional mutual information maximization criterion (CMIM). The new criterion, called CMIM-2, allows detecting relevant features that are complementary in the class prediction better than the original criterion. In addition, we present a methodology to approximate the conditional mutual information to spaces of three variables, avoiding its estimation in high-dimensional spaces. Experimental results for artificial and UCI benchmark datasets show that the proposed criterion outperforms the original CMIM criterion.

Słowa kluczowe

EN

feature selection; conditional mutual information; information theory; relevance; redundancy

• Wydawca: Społeczna Akademia Nauk w Łodzi
• Czasopismo: Journal of Applied Computer Science Methods
• Rocznik: 2010
• Tom: Vol. 2 No. 1
• Strony: 5--20
• Opis fizyczny: Bibliogr. 26 poz., rys., tab.

Twórcy

autor

Vergara J. R.

• Department of Electrical Engineering Universidad de Chile

autor

Estévez P. A.

• Department of Electrical Engineering Universidad de Chile

Bibliografia

• 1. Asuncion A., Newman D., 2007. UCI machine learning repository. http://www.ics.uci.edu/∼mlearn/MLRepository.html, University of California, Irvine, School of Information and Computer Sciences.
• 2. Battiti R., 1994, Using mutual information for selecting features in supervised neural net learning, IEEE Transactions on Neural Networks, Vol. 5, No 4, pp. 537–550.
• 3. Bell D. A., Wang H., 2000, A formalism for relevance and its application in feature subset selection, Machine Learning, Vol. 41, No. 2, pp. 175–195.
• 4. Blum A. L., Rivest R. L., 1992, Training a 3-node neural networks is NP-complete, IEEE Transactions on Neural Networks, Vol. 5, No. 1, pp. 117–127.
• 5. Cover T. M., Thomas J. A., 2006, Elements of Information Theory, 2nd ed. Wi-ley-Interscience.
• 6. Duch W., Winiarski T., Biesiada J., Kachel A., 2003, Feature selection and ranking filter, In International Conference Artificial Neural Networks (ICANN) and International Conference Neural Information Processing (ICONIP), pp. 251–254.
• 7. Estévez P. A., Tesmer M., Pérez C. A., Zurada J. M., 2009, Normalized mutual information feature selection, IEEE Transactions on Neural Networks, Vol. 20, No 2, pp. 189–201.
• 8. Fleuret F., Guyon I., 2004, Fast binary feature selection with conditional mutual information, Journal of Machine Learning Research, Vol. 5, pp. 1531–1555.
• 9. Fraser A. M., Swinney H. L., 1986, Independent coordinates for strange attrac-tors from mutual information, Physical Review A, Vol. 33, No. 2, pp. 1134–1140.
• 10. Guyon I., Elisseeff A., 2003, An introduction to variable and feature selection, Journal of Machine Learning Research, Vol. 3, pp. 1157–1182.
• 11. Hastie T., Tibshiran R., Friedman J., 2001, The Elements of Statistical Learning. Springer.
• 12. Jakulin A., Bratko I., 2003, Quantifying and visualizing attribute interactions, ACM (Computing Research Repository) ,Vol. cs.AI/0308002, pp. –.
• 13. Kohavi R., John G. H., 1997, Wrappers for feature subset selection, Artificial Intelligence Vol. 97, No. 1-2, pp. 273 – 324.
• 14. Koller D., Sahami M., 1996, Toward optimal feature selection, Technical Report 1996-77, Stanford InfoLab.
• 15. Kullback S., 1997, Information Theory and Statistics. New York: Dover.
• 16. Kwak N., Choi C.-H., 2002, Input feature selection for classification problems, IEEE Transactions on Neural Networks, Vol. 13, No. 1, pp. 143–159.
• 17. Liu H., Dougherty E., Dy J., Torkkola K., Tuv E., Peng H., Ding C., Long F., Berens M., Parsons L., Zhao Z., Yu L., Forman G., 2005, Evolving feature selec-tion, IEEE Intelligent Systems, Vol. 20, No. 6, pp. 64–76.
• 18. Meyer P., Schretter C., Bontempi G., 2008, Information-theoretic feature selec-tion in microarray data using variable complementarity, IEEE Journal of Se-lected Topics in Signal Processing, Vol. 2, No. 3, pp. 261–274.
• 19. Peng H., Long F., Ding C., 2005, Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, IEEE Trans-actions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 8, pp. 1226–1238.
• 20. Press W., Flannery B., Teukolsky S., Vetterling W., 1992, Numerical Recipes in C, 2nd ed. Cambridge, U.K.: Cambridge University Press.
• 21. Ripley B. D., 2008, Pattern Recognition and Neural Networks. Cambridge Uni-versity Press.
• 22. Saeys Y., Inza I., Larranaga P., 2007, A review of feature selection techniques in bioinformatics, Bioinformatics, Vol. 23, No. 19, pp. 2507–2517.
• 23. Shannon C. E., 1948, A mathematical theory of communication, Bell System Technical Journal, Vol. 27, pp. 379–423, 625–56.
• 24. Thrun S., Bala J., Bloedorn E., Bratko I., Cestnik B., Cheng J., Jong K. D., Dze-roski S., Hamann R., Kaufman K., Keller S., Kononenko I., Kreuziger J., Mi-chalski R., Mitchell T., Pachowicz P., Roger B., Vafaie H., de Velde W. V., Wenzel W., Wnek J., Zhang J., 1991, The MONK’s problems: A performance comparison of different learning algorithms, Technical Report CMU-CS-91-197, Carnegie Mellon University, Computer Science Department, Pittsburgh, PA.
• 25. Wang G., Lochovsky F. H., 2004, Feature selection with conditional mutual information MaxiMin in text categorization, In proceedings of the thirteenth ACM international conference on information and knowledge management, New York, USA, pp. 342–349.
• 26. Yu L., Liu H., 2004, Efficient feature selection via analysis of relevance and redundancy, Journal Machine Learning Research, Vol. 5, pp. 1205–1224.