Relational Feature Mining with Hierarchical Multitask kFOIL

• Autorzy: Cilia E. , Landwehr N. , Passerini A.
• Języki publikacji: EN

Abstrakty

EN

We introduce hierarchical kFOIL as a simple extension of the multitask kFOIL learning algorithm. The algorithm first learns a core logic representation common to all tasks, and then refines it by specialization on a per-task basis. The approach can be easily generalized to a deeper hierarchy of tasks. A task clustering algorithm is also proposed in order to automatically generate the task hierarchy. The approach is validated on problems of drug-resistance mutation prediction and protein structural classification. Experimental results show the advantage of the hierarchical version over both single and multi task alternatives and its potential usefulness in providing explanatory features for the domain. Task clustering allows to further improve performance when a deeper hierarchy is considered.

Słowa kluczowe

EN

relational learning; hierarchical kFOIL; task clustering

• Wydawca: IOS Press
• Czasopismo: Fundamenta Informaticae
• Rocznik: 2011
• Tom: Vol. 113, nr 2
• Strony: 151--177
• Opis fizyczny: Bibliogr. 43 poz., tab.

Twórcy

autor

Cilia E.

autor

Landwehr N.

autor

Passerini A.

• Departement d’Informatique, Universite Libre de Bruxelles, Belgium,

Bibliografia

• [1] Bakker, B., Heskes, T.: Task clustering and gating for bayesian multitask learning, Journal of Machine Learning Research, 4, 2003, 83-99.
• [2] Baxter, J.: A Bayesian/information theoretic model of learning to learn via multiple task sampling, Machine Learning, 1997, 7-39.
• [3] Bennett, D. E., Camacho, R. J., Otelea, D., Kuritzkes, D. R., Fleury, H., Kiuchi, M., Heneine,W., Kantor, R., Jordan, M. R., Schapiro, J. M., Vandamme, A.-M., Sandstrom, P., Boucher, C. a. B., van de Vijver, D., Rhee, S.-Y., Liu, T. F., Pillay, D., Shafer, R. W.: Drug resistance mutations for surveillance of transmitted HIV-1 drug-resistance: 2009 update., PloS one, 4(3), 2009, e4724.
• [4] Blockeel, H., De Raedt, L., Ramon, J.: Top-down Induction of Clustering Trees, Proceeding of the 15th International Conference on Machine Learning, Madison, Wisconsin, USA, 1998.
• [5] Blockeel, H., Dzeroski, S., Kompare, B., Kramer, S., Pfahringer, B., Laer, W.: Experiments In Predicting Biodegradability., Applied Artificial Intelligence, 18(2), 2004, 157-181.
• [6] Caruana, R.: Multitask Learning, Machine Learning, 28(1), 1997, 41-75.
• [7] Chen, J., Kelley, L., Muggleton, S., Sternberg, M.: Multi-class prediction using stochastic logic programs, Inductive Logic Programming, 2007, 109-124.
• [8] Chen, J., Kelley, L., Muggleton, S., Sternberg, M.: Protein fold discovery using stochastic logic programs, in: Probabilistic inductive logic programming (L. De Raedt, P. Frasconi, K. Kersting, S. Muggleton, Eds.), Springer-Verlag, Berlin, Heidelberg, 2008, 244-262.
• [9] Cristianini, N., Shawe-Taylor, J., Elisseef, A., Kandola, J.: On kernel-target alignment, Proceedings of NIPS 14, 2001.
• [10] Datta, P., Kibler, D. F.: Concept Sharing: A Means to Improve Multi-Concept Learning, Proceedings of the 10th International Conference on Machine Learning, Amherst, MA, USA, 1993.
• [11] De Raedt, L., Lavrac, N., Dzeroski, S.: Multiple Predicate Learning, Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, 1993.
• [12] Deshpande, A., Milch, B., Zettlemoyer, L., Kaelbling, L.: Learning Probabilistic Relational Dynamics for Multiple Tasks., Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI-07), 2007.
• [13] Deshpande, A., Milch, B., Zettlemoyer, L. S., Kaelbling, L. P.: Learning Probabilistic Relational Dynamics for Multiple Tasks, Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI), 2007.
• [14] Evgeniou, T., Micchelli, C. A., Pontil, M.: Learning Multiple Tasks with Kernel Methods, Journal of Machine Learning Research, 6, 2005, 615-637.
• [15] Ferguson, T. S.: A Bayesian Analysis of Some Nonparametric Problems, The Annals of Statistics, 1(2), 1973, 209-230.
• [16] Handel, A., Regoes, R. R., Antia, R.: The Role of Compensatory Mutations in the Emergence of Drug Resistance, PLoS Computational Biology, 2(10), 10 2006, e137.
• [17] Hanley, J., McNeil, B.: A method of comparing the areas under receiver operating characteristic curves derived from the same cases, Radiology, 148(3), 1983, 839-843.
• [18] Hubbard, T., Murzin, A., Brenner, S., Chothia, C.: SCOP: a structural classification of proteins database, Nucleic Acids Research, 25(1), January 1997, 236-9.
• [19] III, H. D.: Bayesian Multitask Learning with Latent Hierarchies, Proceedings of the 25th Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-09), AUAI Press, Corvallis, Oregon, 2009.
• [20] Kemp, C., Tenenbaum, J. B., Griffiths, T. L., Yamada, T., Ueda, N.: Learning systems of concepts with an infinite relational model, AAAI'06: Proceedings of the 21st national conference on Artificial intelligence, AAAI Press, 2006.
• [21] Khan, K., Muggleton, S., Parson, R.: Repeat Learning Using Predicate Invention, Proceedings of Inductive Logic Programming, 8th International Workshop, Madison, Wisconsin, USA, 1446, Springer, 1998.
• [22] Kramer, S., De Raedt, L.: Feature Construction with Version Spaces for Biochemical Applications, Proceedings of the 18th International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., 2001.
• [23] Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: Integrating Naive Bayes and FOIL., Proceedings of the 20th National Conference on Artificial Intelligence, Pittsburgh, Pennsylvania, USA, 2005.
• [24] Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P.: kFOIL: learning simple relational kernels, Proceedings of AAAI'06, 2006.
• [25] Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P.: Fast Learning of Relational Kernels, Machine Learning, 79(3), 2010, 305-342.
• [26] Lengauer, T., Sing, T.: Bioinformatics-assisted anti-HIV therapy, Nature Reviews Microbiology, 4(10), 2006, 790-797.
• [27] Madrid-Sanchez, J., Parrado-Hernandez, E., Figueiras-Vidal, A.: Selective Multitask Learning by Coupling Common and Private Representations, Proceedings of NIPS 08Workshop on Learning from Multiple Sources, 2008.
• [28] Muggleton, S., Raedt, L. U. C. D. E.: Inductive Logic Programming : Theory and Methods, University Computing, 1994, 629-682.
• [29] Neal, R.: Density modeling and clustering using dirichlet diffusion trees, Bayesian Statistics 7, 2003.
• [30] Pettersen, E. F., Goddard, T. D., Huang, C. C., Couch, G. S., Greenblatt, D. M., Meng, E. C., Ferrin, T. E.: UCSF Chimera-a visualization system for exploratory research and analysis., Journal of Computational Chemistry, 25(13), October 2004, 1605-1612, ISSN 0192-8651.
• [31] Quinlan, J.: Learning Logical Definitions from Relations, Machine Learning, 5, 1990, 239-266.
• [32] Reid, M. D.: Improving Rule Evaluation Using Multitask Learning, Proceedings of Inductive Logic Programming, 14th International Conference, Porto, Portugal, 3194, Springer, 2004.
• [33] Rhee, S., Taylor, J., Wadhera, G., Ben-Hur, A.: Genotypic predictors of Human Immunodeficiency Virus type 1 drug resistance, Proceedings of the National Academy of Sciences, Jan 2006.
• [34] Rhee, S.-Y., Gonzales, M. J., Kantor, R., Betts, B. J., Ravela, J., Shafer, R. W.: Human Immunodeficiency Virus reverse transcriptase and protease sequence database, Nucleic Acids Research, 31(1), Jan 2003, 298-303.
• [35] Richter, L., Augustin, R., Kramer, S.: Finding Relational Associations in HIV Resistance Mutation Data, Proceeding of 19th International Conference on Inductive Logic Programming (ILP09), Jun 2009.
• [36] Roy, D. M., Kemp, C., Mansinghka, V. K., Tenenbaum, J. B.: Learning annotated hierarchies from relational data, in: Advances in Neural Information Processing Systems 19 (B. Sch¨olkopf, J. Platt, T. Hoffman, Eds.), MIT Press, Cambridge, MA, 2007, 1185-1192.
• [37] Taylor, W. R.: The classification of amino acid conservation., J Theor Biol, 119(2), March 1986, 205-218.
• [38] Thrun, S., O'Sullivan, J.: Discovering Structure in Multiple Learning Tasks: The TC Algorithm, Proceedings of the International Conference on Machine Learning '96, 1996.
• [39] Turcotte, M., Muggleton, S. H., Sternberg, M. J.: Automated discovery of structural signatures of protein fold and function, Journal of Molecular Biology, 306(3), 2001, 591 - 605, ISSN 0022-2836.
• [40] Wagner, A.: Neutralism and selectionism: a network-based reconciliation., Nature reviews. Genetics, 9(12), December 2008, 965-974.
• [41] Walter, H., Schmidt, B., Werwein, M., Schwingel, E., Korn, K.: Prediction of abacavir resistance from genotypic data: impact of zidovudine and lamivudine resistance in vitro and in vivo, Antimicrobial agents and chemotherapy, 46(1), 2002, 89.
• [42] Xu, Z., Tresp, V., Yu, K., Kriegel, H.: Infinite Hidden Relational Models, Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (UAI 2006), Cambridge, MA, USA, July 2006.
• [43] Xue, Y., Liao, X., Carin, L., Krishnapuram, B.: Multi-Task Learning for Classification with Dirichlet Process Priors, Journal of Machine Learning Research, 8, 2007, 35-63.