Approaching explanatory adequacy in phonology using Minimum Description Length

• Autorzy: Rasin Exer , Berger Iddo , Lan Nur , Shefi Itamar , Katzir Roni

Identyfikatory

DOI

10.15398/jlm.v9i1.266

• Języki publikacji: EN

Abstrakty

EN

A linguistic theory reaches explanatory adequacy if it arrives at a linguistically-appropriate grammar based on the kind of input available to children. In phonology, we assume that children can succeed even when the input consists of surface evidence alone, with no corrections or explicit paradigmatic information – that is, in learning from distributional evidence. We take the grammar to include both a lexicon of underlying representations and a mapping from the lexicon to surface forms. Moreover, this mapping should be able to express optionality and opacity, among other textbook patterns. This learning challenge has not yet been addressed in the literature. We argue that the principle of Minimum Description Length (MDL) offers the right kind of guidance to the learner – favoring generalizations that are neither overly general nor overly specific – and can help the learner overcome the learning challenge. We illustrate with an implemented MDL learner that succeeds in learning various linguistically-relevant patterns from small corpora.

Słowa kluczowe

EN

learning; phonology; opacity; optionality; minimum; Description Lengt

• Wydawca: Instytut Podstaw Informatyki PAN
• Czasopismo: Journal of Language Modelling
• Rocznik: 2021
• Tom: Vol. 9, No. 1
• Strony: 17--66
• Opis fizyczny: Bibliogr. 80 poz., rys., tab.

Twórcy

autor

Rasin Exer

• Department of Linguistics and Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel 6997801

autor

Berger Iddo

• Department of Linguistics and Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel 6997801

autor

Lan Nur

• Department of Linguistics and Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel 6997801

autor

Shefi Itamar

• Department of Linguistics and Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel 6997801

autor

Katzir Roni

• Department of Linguistics and Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel 6997801

Bibliografia

• [1]. Adam ALBRIGHT and Bruce HAYES (2002), Modeling English past tense intuitions with minimal generalization, in Proceedings of the ACL-02 workshop on Morphological and phonological learning-Volume 6, pp. 58-69, Association for Computational Linguistics.
• [2]. Adam ALBRIGHT and Bruce HAYES (2003), Rules vs. analogy in English past tenses: a computational/experimental study, Cognition, 90(2):119-161, doi:http://dx.doi.org/10.1016/S0010-0277(03)00146-X.
• [3]. John ALDERETE and Bruce TESAR (2002), Learning covert phonological interaction: an analysis of the problem posed by the interaction of stress and epenthesis, Technical Report RuCCS-TR-72, Rutgers Center for Cognitive Science, Piscataway, NJ.
• [4]. Carl L. BAKER (1979), Syntactic theory and the projection problem, Linguistic Inquiry, 10(4):533-581.
• [5]. Eric BAKOVIĆ (2011), Opacity and ordering, in The Handbook of Phonological Theory, Second Edition, pp. 40-67, Wiley-Blackwell.
• [6]. Robert C. BERWICK (1982), Locality principles and the acquisition of syntactic knowledge, Ph.D. thesis, MIT, Cambridge, MA.
• [7]. Robert C. BERWICK (1985), The acquisition of syntactic knowledge, MIT Press, Cambridge, Massachusetts.
• [8]. Paul BOERSMA and Bruce HAYES (2001), Empirical Tests of the Gradual Learning Algorithm, Linguistic Inquiry, 32:45-86.
• [9]. Martin D. S. BRAINE (1971), On Two Types of Models of the Internalization of Grammars, in D. J. SLOBIN, editor, The Ontogenesis of Grammar, pp. 153-186, Academic Press.
• [10]. Michael BRENT (1999), An efficient, probabilistically sound algorithm for segmentation and word discovery, Computational Linguistics, 34(1-3):71-105.
• [11]. Shira CALAMARO and Gaja JAROSZ (2015), Learning General Phonological Rules From Distributional Information: A Computational Model, Cognitive Science, 39(3):647-666, doi:10.1111/cogs.12167.
• [12]. Gregory J. CHAITIN (1966), On the Length of Programs for Computing Finite Binary Sequences, Journal of the ACM, 13:547-569.
• [13]. Noam CHOMSKY (1964), Current issues in linguistic theory, Mouton & Company.
• [14]. Noam CHOMSKY (1965), Aspects of the theory of syntax, MIT Press, Cambridge, MA.
• [15]. Noam CHOMSKY and Morris HALLE (1968), The Sound Pattern of English, Harper and Row Publishers, New York.
• [16]. Alexander CLARK (2001), Unsupervised Language Acquisition: Theory and Practice, Ph.D. thesis, University of Sussex.
• [17]. Ryan COTTERELL, Nanyun PENG, and Jason EISNER (2015), Modeling Word Forms Using Latent Underlying Morphs and Phonology, Transactions of the Association for Computational Linguistics, 3:433-447.
• [18]. Carl DE MARCKEN (1996), Unsupervised Language Acquisition, Ph.D. thesis, MIT, Cambridge, MA.
• [19]. François DELL (1981), On the learnability of optional phonological rules, Linguistic Inquiry, 12(1):31-37.
• [20]. Kevin ELLIS and Timothy O’DONNELL (2017), Inducing phonological rules: Perspectives from Bayesian program learning, Presented at the MIT Workshop on Simplicity in Grammar Learning.
• [21]. Timothy Mark ELLISON (1994), The machine learning of phonological structure, Ph.D. thesis, University of Western Australia.
• [22]. Ansgar D. ENDRESS and Marc D. HAUSER (2011), The influence of type and token frequency on the acquisition of affixation patterns: Implications for language processing., Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(1):77-95.
• [23]. Wallace M. ERWIN (1963), A short reference grammar of Iraqi Arabic, Georgetown University Press.
• [24]. Louann GERKEN, Rachel WILSON, and William LEWIS (2005), Infants can use distributional cues to form syntactic categories, Journal of Child Language, 32(2):249-268.
• [25]. John GOLDSMITH (2001), Unsupervised Learning of the Morphology of a Natural Language, Computational Linguistics, 27(2):153-198.
• [26]. John GOLDSMITH (2006), An Algorithm for the Unsupervised Learning of Morphology, Natural Language Engineering, 12(3):1-19.
• [27]. Sharon. GOLDWATER, Thomas L. GRIFFITHS, and Mark JOHNSON (2006), Interpolating between types and tokens by estimating power-law generators, Advances in neural information processing systems, 18:459.
• [28]. Sharon GOLDWATER and Mark JOHNSON (2004), Priors in Bayesian Learning of Phonological Rules, in 7th Annual Meeting of the ACL Special Interest Group on Computational Phonology, pp. 35-42.
• [29]. N.D. GOODMAN, J.B. TENENBAUM, J. FELDMAN, and T.L. GRIFFITHS (2008), A Rational Analysis of Rule-Based Concept Learning, Cognitive Science, 32(1):108-154.
• [30]. Peter GRÜNWALD (1996), A Minimum Description Length Approach to Grammar Inference, in Stefan WERMTER, Ellen RILOFF, and Gabriele SCHELER, editors, Connectionist, Statistical and Symbolic Approaches to Learning for Natural Approaching explanatory adequacy using MDL Language Processing, Springer Lecture Notes in Artificial Intelligence, pp. 203-216, Springer.
• [31]. Mark HALE and Charles REISS (2003), The Subset Principle in phonology: why the tabula can’t be rasa, Journal of Linguistics, 39:219-244.
• [32]. Mark HALE and Charles REISS (2008), The phonological enterprise, Oxford University Press.
• [33]. John H. HOLLAND (1975), Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence., University of Michigan Press.
• [34]. James HORNING (1969), A Study of Grammatical Inference, Ph.D. thesis, Stanford University.
• [35]. Gaja JAROSZ (2006), Rich Lexicons and Restrictive Grammars – Maximum Likelihood Learning in Optimality Theory, Ph.D. thesis, Johns Hopkins University, Baltimore, Maryland.
• [36]. Gaja JAROSZ (2009), Restrictiveness in Phonological Grammar and Lexicon Learning, in Proceedings from the Annual Meeting of the Chicago Linguistic Society, volume 43, pp. 125-139, Chicago Linguistic Society.
• [37]. Mark JOHNSON (1984), A Discovery Procedure for Certain Phonological Rules, in Proceedings of 10th International Conference on Computational Linguistics and 22nd Annual Meeting of the Association for Computational Linguistics, pp. 344-347.
• [38]. Ronald M. KAPLAN and Martin KAY (1994), Regular Models of Phonological Rule Systems, Computational Linguistics, 20(3):331-378.
• [39]. Roni KATZIR (2014), A Cognitively Plausible Model for Grammar Induction, Journal of Language Modelling, 2(2):213-248.
• [40]. Roni KATZIR, Nur LAN, and Noa PELED (2020), A note on the representation and learning of quantificational determiners, in Michael FRANKE et al., editors, Proceedings of Sinn und Bedeutung 24, volume 1, pp. 392-410.
• [41]. Paul KIPARSKY (1971), Historical linguistics, in W. O. DINGWALL, editor, A Survey of Linguistic Science, pp. 576-642, University of Maryland Linguistics Program, College Park.
• [42]. Paul KIPARSKY (2000), Opacity and Cyclicity, The Linguistic Review, 17(2-4):351-366, doi:10.1515/tlir.2000.17.2-4.351.
• [43]. Andrei Nikolaevic KOLMOGOROV (1965), Three Approaches to the Quantitative Definition of Information, Problems of Information Transmission (Problemy Peredachi Informatsii), 1:1-7, republished as Kolmogorov (1968).
• [44]. Andrei Nikolaevic KOLMOGOROV (1968), Three Approaches to the Quantitative Definition of Information, International Journal of Computer Mathematics, 2:157-168.
• [45]. Martin KRÄMER (2012), Underlying Representations, Cambridge University Press, Cambridge, UK.
• [46]. Nur LAN (2018), Learning morpho-phonology using the Minimum Description Length Principle and a Genetic Algorithm, Master’s thesis, Tel Aviv University.
• [47]. Joan MASCARÓ (1976), Catalan Phonology and the Phonological Cycle, Ph.D. thesis, MIT.
• [48]. John J. MCCARTHY (2005), Taking a free ride in morphophonemic learning, Catalan Journal of Linguistics, 4:19-56.
• [49]. Elliott MORETON and Joe PATER (2012a), Structure and Substance in Artificial-phonology Learning, Part I: Structure, Language and Linguistics Compass, 6(11):686-701.
• [50]. Elliott MORETON and Joe PATER (2012b), Structure and Substance in Artificial-Phonology Learning, Part II: Substance, Language and Linguistics Compass, 6(11):702-718.
• [51]. Elliott MORETON, Joe PATER, and Katya PERTSOVA (2017), Phonological Concept Learning, Cognitive Science, 41(1):4-69.
• [52]. Jason NARADOWSKY and Sharon GOLDWATER (2009), Improving Morphology Induction by Learning Spelling Rules, in IJCAI, pp. 1531-1536.
• [53]. Aleksei NAZAROV and Joe PATER (2017), Learning opacity in Stratal Maximum Entropy Grammar, Phonology, 34(2):299-324.
• [54]. Andrew NEVINS and Bert VAUX (2007), Underlying representations that do not minimize grammatical violations, in Sylvia BLAHO, Patrik BYE, and Martin KRÄMER, editors, Freedom of analysis?, pp. 35-61, Mouton de Gruyter.
• [55]. Charlie O’HARA (2017), How abstract is more abstract? Learning abstract underlying representations, Phonology, 34(2):325-345.
• [56]. Gergő ORBÁN, József FISER, Richard N. ASLIN, and Máté LENGYEL (2008), Bayesian learning of visual chunks by human observers, Proceedings of the National Academy of Sciences, 105(7):2745-2750.
• [57]. Sharon PEPERKAMP, Rozenn Le CALVEZ, Jean-Pierre NADAL, and Emmanuel DUPOUX (2006), The acquisition of allophonic rules: Statistical learning with linguistic constraints, Cognition, 101(3):B31-B41, doi:http://dx.doi.org/10.1016/j.cognition.2005.10.006.
• [58]. Alan PRINCE and Paul SMOLENSKY (1993), Optimality Theory: Constraint Interaction in Generative Grammar, Technical report, Rutgers University, Center for Cognitive Science.
• [59]. Alan PRINCE and Bruce TESAR (2004), Learning phonotactic distributions, in René KAGER, Joe PATER, and Wim ZONNEVELD, editors, Constraints in phonological acquisition, pp. 245-291, Cambridge University Press.
• [60]. Anne PYCHA, Pawel NOWAK, Eurie SHIN, and Ryan SHOSTED (2003), Phonological rule-learning and its implications for a theory of vowel harmony, in Proceedings of the 22nd West Coast Conference on Formal Linguistics, pp. 101-114, Cascadilla Press, Somerville, MA.
• [61]. Ezer RASIN and Roni KATZIR (2016), On evaluation metrics in Optimality Theory, Linguistic Inquiry, 47(2):235-282, doi:10.1162/ling_a_00210.
• [62]. Ezer RASIN and Roni KATZIR (2018), Learning abstract underlining representations from distributional evidence, in S. HUCKLEBRIDGE and M. NELSON, editors, Proceedings of NELS 48, pp. 283-290, doi:10.1017/S0022226720000146.
• [63]. Ezer RASIN and Roni KATZIR (2020), A Conditional Learnability Argument for Constraints on Underlying Representations, Journal of Linguistics, 56(4):745-773.
• [64]. Jason RIGGLE (2006), Using entropy to learn OT grammars from surface forms alone, in Proceedings of the 25th West Coast Conference on Formal Linguistics, pp. 346-353.
• [65]. Jorma RISSANEN (1978), Modeling by Shortest Data Description, Automatica, 14:465-471.
• [66]. Jorma RISSANEN and Eric Sven RISTAD (1994), Language Acquisition in the MDL Framework, in Language computations: DIMACS Workshop on Human Language, March 20-22, 1992, p. 149, Amer Mathematical Society.
• [67]. Jenny R. SAFFRAN, Elissa L. NEWPORT, and Richard N. ASLIN (1996), Statistical learning by 8-month old infants, Science, 274:1926-1928.
• [68]. Marc SIMPSON (2010), From alternations to ordered rules: A system for learning Derivational Phonology, Master’s thesis, Concordia University, Montreal.
• [69]. Paul SMOLENSKY (1996), The initial state and ‘richness of the base’ in Optimality Theory, Technical Report JHU-CogSci-96-4, Johns Hopkins University.
• [70]. Ray J. SOLOMONOFF (1964a), A formal theory of inductive inference, part I, Information and Control, 7(1):1-22.
• [71]. Ray J. SOLOMONOFF (1964b), A formal theory of inductive inference, part II, Information and Control, 7(2):224-254.
• [72]. Andreas STOLCKE (1994), Bayesian Learning of Probabilistic Language Models, Ph.D. thesis, University of California at Berkeley, Berkeley, California.
• [73]. Bruce TESAR (1995), Computational optimality theory, Ph.D. thesis, University of Colorado.
• [74]. Bruce TESAR (2014), Output-Driven Phonology, Cambridge University Press.
• [75]. Bruce TESAR and Paul SMOLENSKY (1998), Learnability in Optimality Theory, Linguistic Inquiry, 29(2):229-268.
• [76]. Bert VAUX (2016), Can epenthesis counterbleed assimilation?, talk presented at NAPhC 9, Concordia University, May 7-8, 2016.
• [77]. Christopher S. WALLACE and David M. BOULTON (1968), An Information Measure for Classification, Computer Journal, 11(2):185-194.
• [78]. Kenneth WEXLER and Rita M. MANZINI (1987), Parameters and Learnability in Binding Theory, in Thomas ROEPER and Edwin WILLIAMS, editors, Parameter Setting, pp. 41-76, D. Reidel Publishing Company, Dordrecht, The Netherlands.
• [79]. Katherine S. WHITE, Sharon PEPERKAMP, Cecilia KIRK, and James L. MORGAN (2008), Rapid acquisition of phonological alternations by infants, Cognition, 107(1):238-265.
• [80]. Charles YANG (2016), The Price of Linguistic Productivity: How Children Learn to Break the Rules of Language, MIT Press.