Tytuł artykułu
Autorzy
Wybrane pełne teksty z tego czasopisma
Identyfikatory
Warianty tytułu
Języki publikacji
Abstrakty
Recently, different theoretical learning results have been found for a variety of contextfree grammar subclasses through the use of distributional learning [1]. However, these results are still not extended to probabilistic grammars. In this work, we give a practical algorithm, with some proven properties, that learns a subclass of probabilistic grammars from positive data. A minimum satisfiability solver is used to direct the search towards small grammars. Experiments on well-known context-free languages and artificial natural language grammars give positive results. Moreover, our analysis shows that the type of grammars induced by our algorithm are, in theory, capable of modelling context-free features of natural language syntax. One of our experiments shows that our algorithm can potentially outperform the state-of-the-art in unsupervised parsing on the WSJ10 corpus.
Wydawca
Czasopismo
Rocznik
Tom
Strony
379--402
Opis fizyczny
Bibliogr. 93 poz., rys., tab.
Twórcy
autor
- Université de Nantes, CNRS, LINA, UMR6241 F-44000, France
autor
- Université de Nantes, CNRS, LINA, UMR6241 F-44000, France
Bibliografia
- [1] Clark A. Towards General Algorithms for Grammatical Inference. In: Hutter M, Stephan F, Vovk V, Zeugmann T, editors. ALT. vol. 6331 of Lecture Notes in Computer Science. Springer; 2010. p. 11-30. Available from: http://dx.doi.org/10.1007/978-3-642-16108-7_2.
- [2] Clark A, Eyraud R. Polynomial Identification in the Limit of Substitutable Context-free Languages. Journal of Machine Learning Research. 2007;8:1725-1745. Available from: http://dl.acm.org/citation.cfm?id=1314556.
- [3] Yoshinaka R. Identification in the Limit of k, l-Substitutable Context-Free Languages. In: Clark A, Coste F, Miclet L, editors. ICGI. vol. 5278 of Lecture Notes in Computer Science. Springer; 2008. p. 266-279. Available from: http://dx.doi.org/10.1007/978-3-540-88009-7_21.
- [4] Clark A. PAC-Learning Unambiguous NTS Languages. In: Sakakibara Y, Kobayashi S, Sato K, Nishino T, Tomita E, editors. ICGI. vol. 4201 of Lecture Notes in Computer Science. Springer; 2006. p. 59-71. Available from: http://dx.doi.org/10.1007/11872436_6.
- [5] Clark A. Distributional Learning of Some Context-Free Languages with a Minimally Adequate Teacher. In: Sempere JM, García P, editors. ICGI. vol. 6339 of Lecture Notes in Computer Science. Springer; 2010. p. 24-37. Available from: http://dx.doi.org/10.1007/978-3-642-15488-1_4.
- [6] Shibata C, Yoshinaka R. PAC Learning of Some Subclasses of Context-Free Grammars with Basic Distributional Properties from Positive Data. In: Jain S, Munos R, Stephan F, Zeugmann T, editors. ALT. vol. 8139 of Lecture Notes in Computer Science. Springer; 2013. p. 143-157. Available from: http://dx.doi.org/10.1007/978-3-642-40935-6_11.
- [7] Clark A. Learning Trees from Strings: A Strong Learning Algorithm for some Context-Free Grammars. Journal of Machine Learning Research. 2013;14:3537-3559. Available from: http://dl.acm.org/citation.cfm?id=2567709.2627670.
- [8] Lari K, Young SJ. The estimation of stochastic context-free grammars using the Inside-Outside algorithm. Computer Speech & Language. 1990;4(1):35-56. doi: 10.1016/0885-2308(90)90022-X.
- [9] Carroll G, Charniak E. Two experiments on learning probabilistic dependency grammars from corpora. Department of Computer Science, Univ.; 1992.
- [10] Shibata C, Yoshinaka R. Probabilistic learnability of context-free grammars with basic distributional properties from positive examples. Theoretical Computer Science 620, 2015. doi: 10.1016/j.tcs.2015.10.037.
- [11] Wetherell CS. Probabilistic Languages: A Review and Some Open Questions. ACM Comput Surv. 1980;12(4):361-379. Available from: http://doi.acm.org/10.1145/356827.356829.
- [12] Bartol W, Miró J, Pióro K, Rosselló F. On the coverings by tolerance classes. Inf Sci. 2004;166(1-4):193-211. Available from: http://dx.doi.org/10.1016/j.ins.2003.12.002.
- [13] Boasson L, Sénizergues G. NTS Languages Are Deterministic and Congruential. J Comput Syst Sci. 1985;31(3):332-342. Available from: http://dx.doi.org/10.1016/0022-0000(85)90056-X.
- [14] Gallo G, Longo G, Pallottino S. Directed Hypergraphs and Applications. Discrete Applied Mathematics. 1993;42(2):177-201. doi:10.1016/0166-218X(93)90045-P.
- [15] de la Higuera C. Grammatical Inference: Learning Automata and Grammars. New York, USA: Cambridge University Press; 2010. ISBN: 0521763169, 9780521763165.
- [16] Stolcke A. Bayesian learning of probabilistic language models. University of California, Berkeley; 1994.
- [17] Borensztajn G, Zuidema W. Bayesian model merging for unsupervised constituent labeling and grammar induction. Inst. for Logic, Language and Computation; 2007.
- [18] Rytter W. Application of Lempel-Ziv factorization to the approximation of grammar-based compression. Theor Comput Sci. 2003;302(1-3):211-222. Available from: http://dx.doi.org/10.1016/S0304-3975(02)00777-6.
- [19] Charikar M, Lehman E, Liu D, Panigrahy R, Prabhakaran M, Sahai A, et al. The smallest grammar problem. IEEE Transactions on Information Theory. 2005;51(7):2554-2576. Available from: http://dx.doi.org/10.1109/TIT.2005.850116.
- [20] Marathe MV, Ravi SS. On Approximation Algorithms for the Minimum Satisfiability Problem. Inf Process Lett. 1996;58(1):23-29. Available from: http://dx.doi.org/10.1016/0020-0190(96)00031-2.
- [21] Solan Z, Horn D, Ruppin E, Edelman S. Unsupervised learning of natural languages. Proceedings of the National Academy of Sciences of the United States of America. 2005;102(33):11629-11634.
- [22] Waterfall HR, Sandbank B, Onnis L, Edelman S. An empirical generative framework for computational modeling of language acquisition. Journal of Child Language. 2010 6;37:671-703.
- [23] van Zaanen M. Bootstrapping Structure into Language: Alignment-Based Learning. University of Leeds; 2001.
- [24] Langley P, Stromsten S. Learning Context-Free Grammars with a Simplicity Bias. In: de Mántaras RL, Plaza E, editors. ECML. vol. 1810 of Lecture Notes in Computer Science. Springer; 2000. p. 220-228. Available from: http://dx.doi.org/10.1007/3-540-45164-1_23.
- [25] Adriaans PW, Trautwein M, Vervoort M. Towards High Speed Grammar Induction on Large Text Corpora. In: Hlavác V, Jeffery KG, Wiedermann J, editors. SOFSEM. vol. 1963 of Lecture Notes in Computer Science. Springer; 2000. p. 173-186. Available from: http://dx.doi.org/10.1007/3-540-44411-4_11.
- [26] van Zaanen M, Geertzen J. Problems with Evaluation of Unsupervised Empirical Grammatical Inference Systems. In: Clark A, Coste F, Miclet L, editors. ICGI. vol. 5278 of Lecture Notes in Computer Science. Springer; 2008. p. 301-303. Available from: http://dx.doi.org/10.1007/978-3-540-88009-7_29.
- [27] Luque FM, López GGI. Bounding the Maximal Parsing Performance of Non-Terminally Separated Grammars. In: Sempere JM, García P, editors. ICGI. vol. 6339 of Lecture Notes in Computer Science. Springer; 2010. p. 135-147. Available from: http://dx.doi.org/10.1007/978-3-642-15488-1_12.
- [28] Klein D. The Unsupervised Learning of Natural Language Structure. PhD thesis, Stanford University; 2004.
- [29] Bod R. Unsupervised parsing with U-DOP. In: Proceedings of the Tenth Conference on Computational Natural Language Learning. CoNLL-X ’06. Stroudsburg, PA, USA: Association for Computational Linguistics; 2006. p. 85-92.
- [30] Bod R. An all-subtrees approach to unsupervised parsing. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2006. p. 865-872.
- [31] Seginer Y. Fast Unsupervised Incremental Parsing. In: Carroll JA, van den Bosch A, Zaenen A, editors. ACL; 2007. Available from: http://aclweb.org/anthology-new/P/P07/P07-1049.pdf.
- [32] Yoshinaka R, Clark A. Polynomial Time Learning of Some Multiple Context-Free Languages with a Minimally Adequate Teacher. In: de Groote P, Nederhof MJ, editors. FG. vol. 7395 of Lecture Notes in Computer Science. Springer; 2010. p. 192-207. Available from: http://dx.doi.org/10.1007/978-3-642-32024-8_13.
- [33] Kasprzik A, Yoshinaka R. Distributional Learning of Simple Context-Free Tree Grammars. In: Kivinen J, Szepesvári C, Ukkonen E, Zeugmann T, editors. ALT. vol. 6925 of Lecture Notes in Computer Science. Springer; 2011. p. 398-412. Available from: http://dx.doi.org/10.1007/978-3-642-24412-4_31.
- [34] Clark A. Unsupervised Language Acquisition: Theory and Practice. COGS, University of Sussex; 2001.
- [35] Petasis G, Paliouras G, Spyropoulos CD, Halatsis C. eg-GRIDS: Context-Free Grammatical Inference from Positive Examples Using Genetic Search. In: Paliouras G, Sakakibara Y, editors. ICGI. vol. 3264 of Lecture Notes in Computer Science. Springer; 2004. p. 223-234. Available from: http://dx.doi.org/10.1007/978-3-540-30195-0_20.
- [36] Clark A, Coste F, Miclet L, editors. Grammatical Inference: Algorithms and Applications, 9th International Colloquium, ICGI 2008, Saint-Malo, France, September 22-24, 2008, Proceedings. vol. 5278 of Lecture Notes in Computer Science. Springer; 2008.
- [37] Lang KJ, Pearlmutter BA, Price RA. Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm. In: Honavar V, Slutzki G, editors. ICGI. vol. 1433 of Lecture Notes in Computer Science. Springer; 1998. p. 1-12. Available from: http://dx.doi.org/10.1007/BFb0054059.
- [38] Honavar V, Slutzki G, editors. Grammatical Inference, 4th International Colloquium, ICGI-98, Ames, Iowa, USA, July 12-14, 1998, Proceedings. vol. 1433 of Lecture Notes in Computer Science. Springer; 1998.
- [39] Johnson M, Griffiths TL, Goldwater S. Bayesian Inference for PCFGs via Markov Chain Monte Carlo. In: Sidner CL, Schultz T, Stone M, Zhai C, editors. HLT-NAACL. The Association for Computational Linguistics; 2007. p. 139-146. Available from: http://www.aclweb.org/anthology/N07-1018.
- [40] Sidner CL, Schultz T, Stone M, Zhai C, editors. Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, April 22-27, 2007, Rochester, New York, USA. The Association for Computational Linguistics; 2007.
- [41] Aho AV, Ullman JD. The Theory of Parsing, Translation and Compiling. Englewood Cliffs, NJ: Prentice-Hall; 1972. Volume 1.
- [42] Association AP. Publications Manual. Washington, DC: American Psychological Association; 1983.
- [43] for Computing Machinery A. Computing Reviews. 1983;24(11):503-512.
- [44] Hlavác V, Jeffery KG, Wiedermann J, editors. SOFSEM 2000: Theory and Practice of Informatics, 27th Conference on Current Trends in Theory and Practice of Informatics, Milovy, Czech Republic, November 25-December 2, 2000, Proceedings. vol. 1963 of Lecture Notes in Computer Science. Springer; 2000.
- [45] Bordag S, Hänig C, Quasthoff U. UnsuParse: unsupervised Parsing with unsupervised Part of Speech Tagging. In: European, editor. Proceedings of the Sixth International Language Resources and Evaluation (LREC’08). Marrakech, Morocco; 2008.
- [46] Carroll JA, van den Bosch A, Zaenen A, editors. ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic; 2007.
- [47] Sempere JM, García P, editors. Grammatical Inference: Theoretical Results and Applications, 10th International Colloquium, ICGI 2010, Valencia, Spain, September 13-16, 2010. Proceedings. vol. 6339 of Lecture Notes in Computer Science. Springer; 2010.
- [48] Khanna S, Sudan M, Trevisan L, Williamson DP. The Approximability of Constraint Satisfaction Problems. SIAM J Comput. 2000;30(6):1863-1920. Available from: http://dx.doi.org/10.1137/S0097539799349948.
- [49] de Mántaras RL, Plaza E, editors. Machine Learning: ECML 2000, 11th European Conference on Machine Learning, Barcelona, Catalonia, Spain, May 31-June 2, 2000, Proceedings. vol. 1810 of Lecture Notes in Computer Science. Springer; 2000.
- [50] Paliouras G, Sakakibara Y, editors. Grammatical Inference: Algorithms and Applications, 7th International Colloquium, ICGI 2004, Athens, Greece, October 11-13, 2004, Proceedings. vol. 3264 of Lecture Notes in Computer Science. Springer; 2004.
- [51] Rauh G. Syntactic Categories: Their Identification and Description in Linguistic Theories. Oxford Surveys in Syntax & Morphology No.7. OUP Oxford; 2010.
- [52] Clark A, Lappin S. Unsupervised Learning and Grammar Induction. In: Clark A, Fox C, Lappin S, editors. The Handbook of Computational Linguistics and Natural Language Processing. Wiley-Blackwell; 2010. p. 197-220.
- [53] Yang C. Computational models of syntactic acquisition. Wiley Interdisciplinary Reviews: Cognitive Science. 2012;3(2):205-213.
- [54] de la Higuera C. Characteristic Sets for Polynomial Grammatical Inference. Machine Learning. 1997;27(2):125-138.
- [55] Horning JJ. A Study of Grammatical Inference; 1969.
- [56] Gold EM. Language Identification in the Limit. Information and Control. 1967;10(5):447-474.
- [57] Manning CD, Schütze H. Foundations of statistical natural language processing. MIT Press; 2001.
- [58] Jurafsky D, Martin JH. Speech and Language Processing (2nd Edition) (Prentice Hall Series in Artificial Intelligence). 2nd ed. Prentice Hall; 2008.
- [59] Merlo P, Bunt H, Nivre J. Current Trends in Parsing Technology. In: Trends in Parsing Technology. Springer; 2010. p. 1-17.
- [60] Berkelaar M, et al. lpSolve: Interface to Lp solve v. 5.5 to solve linear/integer programs. R package version. 2008;5(4).
- [61] Hutter M, Stephan F, Vovk V, Zeugmann T, editors. Algorithmic Learning Theory, 21st International Conference, ALT 2010, Canberra, Australia, October 6-8, 2010. Proceedings. vol. 6331 of Lecture Notes in Computer Science. Springer; 2010. Available from: http://dx.doi.org/10.1007/978-3-642-16108-7.
- [62] Jelinek F, Lafferty JD. Computation of the Probability of Initial Substring Generation by Stochastic Context-Free Grammars. Computational Linguistics. 1991;17(3):315-323.
- [63] Stolcke A. An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities. Computational Linguistics. 1995;21(2):165-201.
- [64] Stolcke A, Segal J. Precise N-Gram Probabilities from Stochastic Context-Free Grammars. In: Pustejovsky J, editor. ACL. Morgan Kaufmann Publishers / ACL; 1994. p. 74-79. Available from: http://aclweb.org/anthology-new/P/P94/.
- [65] Pustejovsky J, editor. 32nd Annual Meeting of the Association for Computational Linguistics, 27-30 June 1994, New Mexico State University, Las Cruces, New Mexico, USA, Proceedings. Morgan Kaufmann Publishers / ACL; 1994. Available from: http://aclweb.org/anthology-new/P/P94/.
- [66] Booth TL, Thompson RA. Applying Probability Measures to Abstract Languages. IEEE Trans Comput. 1973 May;22(5):442-450. doi:10.1109/T-C.1973.223746.
- [67] de la Higuera C, Oncina J. Learning Stochastic Finite Automata. In: Paliouras G, Sakakibara Y, editors. ICGI. vol. 3264 of Lecture Notes in Computer Science. Springer; 2004. p. 175-186. Available from: http://dx.doi.org/10.1007/978-3-540-30195-0_16.
- [68] Yoshinaka R. Polynomial-Time Identification of an Extension of Very Simple Grammars from Positive Data. In: Sakakibara Y, Kobayashi S, Sato K, Nishino T, Tomita E, editors. ICGI. vol. 4201 of Lecture Notes in Computer Science. Springer; 2006. p. 45-58. Available from: http://dx.doi.org/10.1007/11872436_5.
- [69] Laxminarayana JA, Nagaraja G. Inference of a Subclass of Context Free Grammars Using Positive Samples. In: de la Higuera C, Adriaans PW, van Zaanen M, Oncina J, editors. ECML Workshop on Learning Contex-Free Grammars. Ruder Boskovic Institute, Zagreb, Croatia; 2003. p. 29-40.
- [70] de la Higuera C, Adriaans PW, van Zaanen M, Oncina J, editors. Proceedings of theWorkshop and Tutorial on Learning Contex-Free Grammars, ECML 2003, Cavtat-Dubrovnik, Croatia, September 22-26, 2003. Ruder Boskovic Institute, Zagreb, Croatia; 2003.
- [71] Yokomori T. Polynomial-time identification of very simple grammars from positive data. Theor Comput Sci. 2003;1(298):179-206. Available from: http://dx.doi.org/10.1016/S0304-3975(02)00423-1.
- [72] de la Higuera C, Oncina J. Inferring Deterministic Linear Languages. In: Kivinen J, Sloan RH, editors. COLT. vol. 2375 of Lecture Notes in Computer Science. Springer; 2002. p. 185-200. Available from: http://dx.doi.org/10.1007/3-540-45435-7_13.
- [73] Kivinen J, Sloan RH, editors. Computational Learning Theory, 15th Annual Conference on Computational Learning Theory, COLT 2002, Sydney, Australia, July 8-10, 2002, Proceedings. vol. 2375 of Lecture Notes in Computer Science. Springer; 2002.
- [74] Eyraud R, de la Higuera C, Janodet JC. LARS: A learning algorithm for rewriting systems. Machine Learning. 2007;66(1):7-31. Available from: http://dx.doi.org/10.1007/s10994-006-9593-8.
- [75] Sempere JM, García P. A Characterization of Even Linear Languages and its Application to the Learning Problem. In: Carrasco RC, Oncina J, editors. ICGI. vol. 862 of Lecture Notes in Computer Science. Springer; 1994. p. 38-44. Available from: http://dx.doi.org/10.1007/3-540-58473-0_135.
- [76] Carrasco RC, Oncina J, editors. Grammatical Inference and Applications, Second International Colloquium, ICGI-94, Alicante, Spain, September 21-23, 1994, Proceedings. vol. 862 of Lecture Notes in Computer Science. Springer; 1994.
- [77] de la Higuera C, Oncina J. Identification with Probability One of Stochastic Deterministic Linear Languages. In: Gavaldà R, Jantke KP, Takimoto E, editors. ALT. vol. 2842 of Lecture Notes in Computer Science. Springer; 2003. p. 247-258. Available from: http://dx.doi.org/10.1007/978-3-540-39624-6_20.
- [78] Gavaldà R, Jantke KP, Takimoto E, editors. Algorithmic Learning Theory, 14th International Conference, ALT 2003, Sapporo, Japan, October 17-19, 2003, Proceedings. vol. 2842 of Lecture Notes in Computer Science. Springer; 2003.
- [79] Ishizaka H. Polynomial Time Learnability of Simple Deterministic Languages. Mach Learn. 1990 Jul;5(2):151-164. doi:10.1023/A:1022644732619.
- [80] Hiromi S, Takashi Y. Polynomial-time MAT Learning of C-Deterministic Context-free Grammars. Journal of Information Processing. 1993 mar;15(EX):42-52.
- [81] Sakakibara Y, Kobayashi S, Sato K, Nishino T, Tomita E, editors. Grammatical Inference: Algorithms and Applications, 8th International Colloquium, ICGI 2006, Tokyo, Japan, September 20-22, 2006, Proceedings. vol. 4201 of Lecture Notes in Computer Science. Springer; 2006.
- [82] Clark A. A Language Theoretic Approach to Syntactic Structure. In: Kanazawa M, Kornai A, Kracht M, Seki H, editors. MOL. vol. 6878 of Lecture Notes in Computer Science. Springer; 2011. p. 39-56. Available from: http://dx.doi.org/10.1007/978-3-642-23211-4_3.
- [83] Kanazawa M, Kornai A, Kracht M, Seki H, editors. The Mathematics of Language - 12th Biennial Conference, MOL 12, Nara, Japan, September 6-8, 2011. Proceedings. vol. 6878 of Lecture Notes in Computer Science. Springer; 2011. Available from: http://dx.doi.org/10.1007/978-3-642-23211-4.
- [84] Angluin D. Learning Regular Sets from Queries and Counterexamples. Inf Comput. 1987;75(2):87-106. Available from: http://dx.doi.org/10.1016/0890-5401(87)90052-6.
- [85] López GGI, de Rijke M. Comparing the Ambiguity Reduction Abilities of Probabilistic Context-Free Grammars. In: LREC. European Language Resources Association; 2004. Available from: http://www.lrec-conf.org/proceedings/lrec2004/pdf/692.pdf.
- [86] Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal. European Language Resources Association; 2004. Available from: http://www.lrec-conf.org/proceedings/lrec2004/.
- [87] Nederhof MJ, Satta G. Computation of distances for regular and context-free probabilistic languages. Theor Comput Sci. 2008;395(2-3):235-254. Available from: http://dx.doi.org/10.1016/j.tcs.2008.01.010.
- [88] Dowell RD, Eddy SR. Evaluation of several lightweight stochastic context-free grammars for RNA secondary structure prediction. BMC Bioinformatics. 2004;5:71. Available from: http://dx.doi.org/10.1186/1471-2105-5-71.
- [89] Sakakibara Y, Brown M, Hughey R, Mian IS, Sjölander K, Underwood RC, et al. Recent Methods for RNA Modeling Using Stochastic Context-Free Grammars. In: Crochemore M, Gusfield D, editors. CPM. vol. 807 of Lecture Notes in Computer Science. Springer; 1994. p. 289-306. Available from: http://dx.doi.org/10.1007/3-540-58094-8_25.
- [90] Crochemore M, Gusfield D, editors. Combinatorial Pattern Matching, 5th Annual Symposium, CPM 94, Asilomar, California, USA, June 5-8, 1994, Proceedings. vol. 807 of Lecture Notes in Computer Science. Springer; 1994.
- [91] de Groote P, Nederhof MJ, editors. Formal Grammar-15th and 16th International Conferences, FG 2010, Copenhagen, Denmark, August 2010, FG 2011, Ljubljana, Slovenia, August 2011, Revised Selected Papers. vol. 7395 of Lecture Notes in Computer Science. Springer; 2012. Available from: http://dx. doi.org/10.1007/978-3-642-32024-8.
- [92] Kivinen J, Szepesvári C, Ukkonen E, Zeugmann T, editors. Algorithmic Learning Theory-22nd International Conference, ALT 2011, Espoo, Finland, October 5-7, 2011. Proceedings. vol. 6925 of Lecture Notes in Computer Science. Springer; 2011. Available from: http://dx.doi.org/10.1007/978-3-642-24412-4.
- [93] Jain S, Munos R, Stephan F, Zeugmann T, editors. Algorithmic Learning Theory - 24th International Conference, ALT 2013, Singapore, October 6-9, 2013. Proceedings. vol. 8139 of Lecture Notes in Computer Science. Springer; 2013. Available from: http://dx.doi.org/10.1007/978-3-642-40935-6.
Typ dokumentu
Bibliografia
Identyfikator YADDA
bwmeta1.element.baztech-51ddd804-e6c4-4674-87c1-1472113c7bf8