Investigating the effects of i-complexity and e-complexity on the learnability of morphological systems

• Autorzy: Johnson Tamar , Gao Kexin , Smith Kenny , Rabagliati Hugh , Culbertson Jennifer

Identyfikatory

DOI

10.15398/jlm.v9i1.259

• Języki publikacji: EN

Abstrakty

EN

Research on cross-linguistic differences in morphological paradigms reveals a wide range of variation on many dimensions, including the number of categories expressed, the number of unique forms, and the number of inflectional classes. However, in an influential paper, Ackerman and Malouf (2013) argue that there is one dimension on which languages do not differ widely: in predictive structure. Predictive structure in a paradigm describes the extent to which forms predict each other, called i-complexity. Ackerman and Malouf (2013) show that although languages differ according to measure of surface paradigm complexity, called e-complexity, they tend to have low i-complexity. They conclude that morphological paradigms have evolved under a pressure for low i-complexity. Here, we evaluate the hypothesis that language learners are more sensitive to i-complexity than e-complexity by testing how well paradigms which differ on only these dimensions are learned. This could result in the typological findings Ackerman and Malouf (2013) report if even paradigms with very high e-complexity are relatively easy to learn, so long as they have low i-complexity. First, we summarize a recent work by Johnson et al. (2020) suggesting that both neural networks and human learners may actually be more sensitive to e-complexity than i-complexity. Then we build on this work, reporting a series of experiments which confirm that, indeed, across a range of paradigms that vary in either e- or icomplexity, neural networks (LSTMs) are sensitive to both, but show a larger effect of e-complexity (and other measures associated with size and diversity of forms). In human learners, we fail to find any effect of i-complexity on learning at all. Finally, we analyse a large number of randomly generated paradigms and show that e- and i-complexity are negatively correlated: paradigms with high e-complexity necessarily show low i-complexity. We discuss what these findings might mean for Ackerman and Malouf’s hypothesis, as well as the role of ease of learning versus generalization to novel forms in the evolution of paradigms.

Słowa kluczowe

EN

morphological; complexity; learning; neural networks; typology

• Wydawca: Instytut Podstaw Informatyki PAN
• Czasopismo: Journal of Language Modelling
• Rocznik: 2021
• Tom: Vol. 9, No. 1
• Strony: 97--150
• Opis fizyczny: Bibliogr. 52 poz., tab., wykr.

Twórcy

autor

Johnson Tamar

• Centre for Language Evolution, University of Edinburgh

autor

Gao Kexin

• Centre for Language Evolution, University of Edinburgh

autor

Smith Kenny

• Centre for Language Evolution, University of Edinburgh

autor

Rabagliati Hugh

• Department of Psychology, University of Edinburgh, Edinburgh, Scotland, United Kingdom

autor

Culbertson Jennifer

• Centre for Language Evolution, University of Edinburgh, Edinburgh, Scotland, United Kingdom

Bibliografia

• [1]. Farrell ACKERMAN, James P. BLEVINS, and Robert MALOUF (2009), Parts and wholes: Patterns of relatedness in complex morphological systems and why they matter, in James P. BLEVINS and Juliette BLEVINS, editors, Analogy in grammar: Form and acquisition, pp. 54-82, Oxford University Press, Oxford.
• [2]. Farrell ACKERMAN and Robert MALOUF (2013), Morphological organization: The low conditional entropy conjecture, Language, 89(3):429-464.
• [3]. Farrell ACKERMAN and Robert MALOUF (2015), The No Blur Principle effects as an emergent property of language systems, in Proceedings of the annual meeting of the Berkeley Linguistics Society, volume 41, pp. 1-14.
• [4]. Ben AMBRIDGE, Evan KIDD, Caroline F. ROWLAND, and Anna L. THEAKSTON (2015), The ubiquity of frequency effects in first language acquisition, Journal of Child Language, 42(2):239-273.
• [5]. Mark ARONOFF (1994), Morphology by itself: Stems and inflectional classes, MIT Press.
• [6]. Matthew BAERMAN, Dunstan BROWN, and Greville G. CORBETT (2005), The syntax-morphology interface: A study of syncretism, Cambridge University Press.
• [7]. Matthew BAERMAN, Dunstan BROWN, and Greville G. CORBETT (2010), Morphological complexity: a typological perspective, https://www.researchgate.net/publication/266215146, unpublished manuscript, University of Surrey.
• [8]. Douglas BATES, Martin MÄCHLER, Ben BOLKER, and Steve WALKER (2014), Fitting linear mixed-effects models using lme4, arXiv preprint arXiv:1406.5823.
• [9]. Balthasar BICKEL and Johanna NICHOLS (2013), Inflectional synthesis of the verb, in Matthew S. DRYER and Martin HASPELMATH, editors, The world atlas of language structures online, Max Planck Institute for Evolutionary Anthropology, https://wals.info/chapter/22.
• [10]. James P. BLEVINS (2006), Word-based morphology, Journal of Linguistics, 42(3):531-573.
• [11]. Olivier BONAMI and Sacha BENIAMINE (2016), Joint predictiveness in inflectional paradigms, Word Structure, 9(2):156-182.
• [12]. François CHOLLET et al. (2015), keras, https://keras.io.
• [13]. Morten H. CHRISTIANSEN and Nick CHATER (2008), Language as shaped by the brain, The Behavioral and Brain Sciences, 31(5):489-509.
• [14]. Greville G. CORBETT (2009), Suppletion: Typology, markedness, complexity, in Patrick O. STEINKRÜGER and Manfred KRIFKA, editors, On inflection, p. 40, Mouton de Gruyter.
• [15]. Ryan COTTERELL, Christo KIROV, Mans HULDEN, and Jason EISNER (2019), On the complexity and typology of inflectional morphological systems, Transactions of the Association for Computational Linguistics, 7:327-342.
• [16]. Jennifer CULBERTSON and Simon KIRBY (2016), Simplicity and specificity in language: Domain-general biases have domain-specific effects, Frontiers in psychology, 6:1964.
• [17]. Jennifer CULBERTSON and Elissa L. NEWPORT (2015), Harmonic biases in child learners: In support of language universals, Cognition, 139(6):71-82.
• [18]. Jennifer CULBERTSON, Paul SMOLENSKY, and Géraldine LEGENDRE (2012), Learning biases predict a word order universal, Cognition, 122(3):306-329.
• [19]. Terrence William DEACON (1997), The symbolic species: The co-evolution of language and the brain, Allen Lane the Penguin Press.
• [20]. Jeffrey L. ELMAN (1990), Finding structure in time, Cognitive Science, 14(2):179-211.
• [21]. Jeffrey L. ELMAN (1991), Distributed representations, simple recurrent networks, and grammatical structure, Machine Learning, 7(2):195-225.
• [22]. Maryia FEDZECHKINA, T. Florian JAEGER, and Elissa L. NEWPORT (2012), Language learners restructure their input to facilitate efficient communication, Proceedings of the National Academy of Sciences, 109(44):17897-17902.
• [23]. Richard FUTRELL, Ethan WILCOX, Takashi MORITA, Peng QIAN, Miguel BALLESTEROS, and Roger LEVY (2019), Neural language models as psycholinguistic subjects: Representations of syntactic state, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1, pp. 32-42.
• [24]. Kristina GULORDAVA, Piotr BOJANOWSKI, Edouard GRAVE, Tal LINZEN, and Marco BARONI (2018), Colorless green recurrent networks dream hierarchically, in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1, pp. 1195-1205.
• [25]. Sepp HOCHREITER and Jürgen SCHMIDHUBER (1997), Long short-term memory, Neural Computation, 9(8):1735-1780.
• [26]. Carla L. HUDSON KAM and Elissa L. NEWPORT (2005), Regularizing unpredictable variation: The roles of adult and child learners in language formation and change, Language Learning and Development, 1(2):151-195.
• [27]. Carla L HUDSON KAM and Elissa L NEWPORT (2009), Getting it right by getting it wrong: When learners change languages, Cognitive Psychology, 59(1):30-66.
• [28]. Tamar JOHNSON, Jennifer CULBERTSON, Hugh RABAGLIATI, and Kenny SMITH (2020), Assessing integrative complexity as a predictor of morphological learning using neural networks and artificial language learning, https://psyarxiv.com/yngw9/, unpublished manuscript, University of Edinburgh.
• [29]. Michael I. JORDAN (1997), Serial order: A parallel distributed processing approach, in John W. DONAHOE and Vivian PACKARD DORSEL, editors, Neural-network models of cognition, pp. 471-495, Elsevier.
• [30]. Charles KEMP and Terry REGIER (2012), Kinship categories across languages reflect general communicative principles, Science, 336(6084):1049-1054.
• [31]. Diederik P. KINGMA and Jimmy BA (2014), Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.
• [32]. Simon KIRBY (2002), Learning, bottlenecks and the evolution of recursive syntax, in Ted BRISCOE, editor, Linguistic evolution through language acquisition, pp. 173-204, Cambridge University Press.
• [33]. Simon KIRBY, Hannah CORNISH, and Kenny SMITH (2008), Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language, Proceedings of the National Academy of Sciences, 105(31):10681-10686.
• [34]. Simon KIRBY, Monica TAMARIZ, Hannah CORNISH, and Kenny SMITH (2015), Compression and communication in the cultural evolution of linguistic structure, Cognition, 141:87-102.
• [35]. Alexandra KUZNETSOVA, Per B. BROCKHOFF, Rune HB CHRISTENSEN, et al. (2017), lmerTest package: tests in linear mixed effects models, Journal of Statistical Software, 82(13):1-26.
• [36]. Tal LINZEN, Emmanuel DUPOUX, and Yoav GOLDBERG (2016), Assessing the ability of LSTMs to learn syntax-sensitive dependencies, Transactions of the Association for Computational Linguistics, 4:521-535.
• [37]. Mora MALDONADO and Jennifer CULBERTSON (2019), Something about ”us”: Learning first person pronoun systems, in Proceedings of the 41st annual meeting of the Cognitive Science Society, pp. 749-755.
• [38]. Robert MALOUF (2017), Abstractive morphological learning with a recurrent neural network, Morphology, 27(4):431-458.
• [39]. Eric MEINHARDT, Rob MALOUF, and Farrell ACKERMAN (2019), Morphology gets more and more complex, unless it doesn’t, https://www.researchgate.net/publication/333194657, unpublished manuscript, San Diego State University and University of California San Diego.
• [40]. Daniel MIRMAN (2017), Growth curve analysis and visualization using R, CRC Press, first edition. edition.
• [41]. Elliott MORETON and Joe PATER (2012), Structure and substance in artificial-phonology learning, part I: Structure, Language and Linguistics Compass, 6(11):686-701.
• [42]. Yasamin MOTAMEDI, Marieke SCHOUWSTRA, Kenny SMITH, Jennifer CULBERTSON, and Simon KIRBY (2019), Evolving artificial sign languages in the lab: From improvised gesture to systematic sign, Cognition, 192:103964-103964.
• [43]. Katya PERTSOVA (2012), Logical complexity in morphological learning: Effects of structure and null/overt affixation on learning paradigms, in Annual meeting of the Berkeley Linguistics Society, volume 38, pp. 401-413.
• [44]. Terry REGIER, Charles KEMP, and Paul KAY (2015), Word meanings across languages support efficient communication, in Brian MACWHINNEY and William O’GRADY, editors, The handbook of language emergence, pp. 237-263, John Wiley & Sons, Inc.
• [45]. Scott SEYFARTH, Farrell ACKERMAN, and Robert MALOUF (2014), Implicative organization facilitates morphological learning, in Annual meeting of the Berkeley Linguistics Society, volume 40, pp. 480-494.
• [46]. Claude Elwood SHANNON (1963), The mathematical theory of communication, University of Illinois Press.
• [47]. Ryan K SHOSTED (2006), Correlating complexity: A typological approach, Linguistic Typology, 10(1):1-40.
• [48]. Catriona SILVEY, Simon KIRBY, and Kenny SMITH (2015), Word meanings evolve to selectively preserve distinctions on salient dimensions, Cognitive Science, 39(1):212-226.
• [49]. Andrea D SIMS and Jeff PARKER (2016), How inflection class systems work: On the informativity of implicative structure, Word Structure, 9(2):215-239.
• [50]. Elizabeth WONNACOTT and Elissa L. NEWPORT (2005), Novelty and regularization: The effect of novel instances on rule formation, in BUCLD 29: Proceedings of the 29th annual Boston University conference on language development, pp. 663-673.
• [51]. Yang XU, Terry REGIER, and Barbara C MALT (2016), Historical semantic chaining and efficient communication: The case of container names, Cognitive Science, 40(8):2081-2094.
• [52]. Noga ZASLAVSKY, Charles KEMP, Naftali TISHBY, and Terry REGIER (2020), Communicative need in colour naming, Cognitive Neuropsychology, 37(5-6):312-324.