Word prediction in computational historical linguistics

• Autorzy: Dekker Peter , Zuidema Willem

Identyfikatory

DOI

10.15398/jlm.v8i2.268

• Języki publikacji: EN

Abstrakty

EN

In this paper, we investigate how the prediction paradigm from machine learning and Natural Language Processing (NLP) can be put to use in computational historical linguistics. We propose word prediction as an intermediate task, where the forms of unseen words in some target language are predicted from the forms of the corresponding words in a source language. Word prediction allows us to develop algorithms for phylogenetic tree reconstruction, sound correspondence identification and cognate detection, in ways close to attested methods for linguistic reconstruction. We will discuss different factors, such as data representation and the choice of machine learning model, that have to be taken into account when applying prediction methods in historical linguistics. We present our own implementations and evaluate them on different tasks in historical linguistics.

Słowa kluczowe

EN

computational historical linguistics; machine learning; deep learning

• Wydawca: Instytut Podstaw Informatyki PAN
• Czasopismo: Journal of Language Modelling
• Rocznik: 2020
• Tom: Vol. 8, No. 2
• Strony: 295--336
• Opis fizyczny: Bibliogr. 90 poz.rys., tab.

Twórcy

autor

Dekker Peter

• AI Lab Vrije. Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium

autor

Zuidema Willem

• Institute for Logic, Language and Computation (ILLC), University of Amsterdam, P.O. Box 94242, 1090 GE Amsterdam, The Netherlands

Bibliografia

• [1]. Yong-Yeol AHN, James P. BAGROW, and Sune LEHMANN (2010), Link Communities Reveal Multiscale Complexity in Networks, Nature, 466(7307):761-764.
• [2]. Enrique AMIGÓ, Julio GONZALO, Javier ARTILES, and Felisa VERDEJO (2009), A Comparison of Extrinsic Clustering Evaluation Metrics Based on Formal Constraints, Information Retrieval, 12(4):461-486.
• [3]. Cormac ANDERSON, Tiago TRESOLDI, Thiago CHACON, Anne-Maria FEHN, Mary WALWORTH, Robert FORKEL, and Johann-Mattis LIST (2018), A Cross-Linguistic Database of Phonetic Transcription Systems, Yearbook of the Poznan Linguistic Meeting, 4(1):21-53, ISSN 2449-7525, doi:10.2478/yplm-2018-0002.
• [4]. Amit BAGGA and Breck BALDWIN (1998), Entity-Based Cross-Document Coreferencing Using the Vector Space Model, in Proceedings of the 17th International Conference on Computational Linguistics, volume 1, pp. 79-85.
• [5]. Dzmitry BAHDANAU, Kyunghyun CHO, and Yoshua BENGIO (2014), Neural Machine Translation by Jointly Learning to Align and Translate, arXiv preprint arXiv:1409.0473.
• [6]. Lisa BEINBORN, Torsten ZESCH, and Iryna GUREVYCH (2013), Cognate Production Using Character-Based Machine Translation, in Ruslan MITKOV and Jong C. PARK, editors, Proceedings of the Sixth International Joint Conference on Natural Language Processing, pp. 883-891, Nagoya, Japan.
• [7]. Yoshua BENGIO, Nicholas LÉONARD, and Aaron COURVILLE (2013), Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation, arXiv:1308.3432 [cs].
• [8]. Timotheus A. BODT and Johann-Mattis LIST (2019), Testing the Predictive Strength of the Comparative Method: An Ongoing Experiment on Unattested Words in Western Kho-Bwa Languages, Papers in Historical Phonology, 4:22-44, ISSN 2399-6714, doi:10.2218/pihph.4.2019.3037.
• [9]. Timotheus A. BODT and Johann-Mattis LIST (2020), The Multiple Benefits of Making Predictions in Linguistics, Babel: The Language Magazine, 31(2):8-12, doi:http://dx.doi.org/10.17613/m688-4b90.
• [10]. Alexandre BOUCHARD-CÔTÉ, David HALL, Thomas L. GRIFFITHS, and Dan KLEIN (2013), Automated Reconstruction of Ancient Languages Using Probabilistic Models of Sound Change, Proceedings of the National Academy of Sciences, 110(11):4224-4229.
• [11]. Alexandre BOUCHARD-CÔTÉ, Percy LIANG, Thomas L. GRIFFITHS, and Dan KLEIN (2007), A Probabilistic Approach to Diachronic Phonology, in Jason EISNER, editor, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp. 887-896.
• [12]. Remco BOUCKAERT, Philippe LEMEY, Michael DUNN, Simon J. GREENHILL, Alexander V. ALEKSEYENKO, Alexei J. DRUMMOND, Russell D. GRAY, Marc A. SUCHARD, and Quentin D. ATKINSON (2012), Mapping the Origins and Expansion of the Indo-European Language Family, Science, 337(6097):957-960.
• [13]. Cecil H. BROWN, Eric W. HOLMAN, Søren WICHMANN, and Viveka VELUPILLAI (2008), Automated Classification of the World’s Languages: A Description of the Method and Preliminary Results, STUF - Language Typology and Universals, 61(4):285-308.
• [14]. David BRYANT, John TSANG, Paul E. KEARNEY, and Ming LI (2000), Computing the Quartet Distance between Evolutionary Trees, in Symposium on Discrete Algorithms: Proceedings of the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms, volume 9, pp. 285-286.
• [15]. Lyle CAMPBELL (2013), Historical Linguistics: An Introduction, MIT Press, second edition.
• [16]. Chundra CATHCART and Taraka RAMA (2020), Disentangling Dialects: A Neural Approach to Indo-Aryan Historical Phonology and Subgrouping, in Raquel FERNÁNDEZ and Tal LINZEN, editors, Proceedings of the 24th Conference on Computational Natural Language Learning, pp. 620-630, Association for Computational Linguistics, Online, doi:10.18653/v1/2020.conll-1.50.
• [17]. Chundra CATHCART and Florian WANDL (2020), In Search of Isoglosses: Continuous and Discrete Language Embeddings in Slavic Historical Phonology, in Garrett NICOLAI, Kyle GORMAN, and Ryan COTTERELL, editors, Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pp. 233-244, Association for Computational Linguistics, Online, doi:10.18653/v1/2020.sigmorphon-1.28.
• [18]. Will CHANG, Chundra CATHCART, David HALL, and Andrew GARRETT (2015), Ancestry-Constrained Phylogenetic Analysis Supports the Indo-European Steppe Hypothesis, Language, 91(1):194-244.
• [19]. Kyunghyun CHO, Bart VAN MERRIËNBOER, Caglar GULCEHRE, Dzmitry BAHDANAU, Fethi BOUGARES, Holger SCHWENK, and Yoshua BENGIO (2014), Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation, in Alessandro MOSCHITTI, Bo PANG, and Walter DAELEMANS, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, Association for Computational Linguistics, Doha, Qatar, doi:10.3115/v1/D14-1179.
• [20]. Kenneth Ward CHURCH and Patrick HANKS (1990), Word Association Norms, Mutual Information, and Lexicography, Computational Linguistics, 16(1):22-29.
• [21]. Alina Maria CIOBANU (2016), Sequence Labeling for Cognate Production, Procedia Computer Science, 96:1391-1399, ISSN 18770509, doi:10.1016/j.procs.2016.08.184.
• [22]. Alina Maria CIOBANU and Liviu P. DINU (2014), Building a Dataset of Multilingual Cognates for the Romanian Lexicon, in Nicoletta CALZOLARI, Khalid CHOUKRI, Thierry DECLERCK, Hrafn LOFTSSON, Bente MAEGAARD, Joseph MARIANI, Asuncion MORENO, Jan ODIJK, and Stelios PIPERIDIS, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation, pp. 1038-1043.
• [23]. Alina Maria CIOBANU and Liviu P. DINU (2018), Ab Initio: Automatic Latin Proto-Word Reconstruction, in Emily M. BENDER, Leon DERCZYNSKI, and Pierre ISABELLE, editors, Proceedings of the 27th International Conference on Computational Linguistics, pp. 1604-1614.
• [24]. Alina Maria CIOBANU and Liviu P. DINU (2020), Automatic Identification and Production of Related Words for Historical Linguistics, Computational Linguistics, 45(4):667-704, ISSN 0891-2017, 1530-9312, doi:10.1162/coli_a_00361.
• [25]. Alina Maria CIOBANU, Liviu P. DINU, and Laurentiu ZOICAS (2020), Automatic Reconstruction of Missing Romanian Cognates and Unattested Latin Words, in Proceedings of the 12th Language Resources and Evaluation Conference, pp. 3226-3231.
• [26]. James CLACKSON (2007), Indo-European Linguistics: An Introduction, Cambridge University Press.
• [27]. Michael COLLINS (2002), Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms, in Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10, pp. 1-8.
• [28]. Matthieu COURBARIAUX, Itay HUBARA, Daniel SOUDRY, Ran EL-YANIV, and Yoshua BENGIO (2016), Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or −1, arXiv:1602.02830 [cs].
• [29]. Harold Charles DAUME and Daniel MARCU (2006), Practical Structured Learning Techniques for Natural Language Processing, University of Southern California.
• [30]. Peter DEKKER (2018), Reconstructing Language Ancestry by Performing Word Prediction with Neural Networks, MSc thesis, University of Amsterdam.
• [31]. Johannes DELLERT (2018), Combining Information-Weighted Sequence Alignment and Sound Correspondence Models for Improved Cognate Detection, in Proceedings of the 27th International Conference on Computational Linguistics, pp. 3123-3133.
• [32]. Johannes DELLERT, Thora DANEYKO, Alla MÜNCH, Alina LADYGINA, Armin BUCH, Natalie CLARIUS, Ilja GRIGORJEW, Mohamed BALABEL, Hizniye Isabella BOGA, Zalina BAYSAROVA, Roland MÜHLENBERND, Johannes WAHLE, and Gerhard JÄGER (2019), NorthEuraLex: A Wide-Coverage Lexical Database of Northern Eurasia, Language Resources and Evaluation, ISSN 1574-020X, 1574-0218, doi:10.1007/s10579-019-09480-6.
• [33]. Rick DERKSEN (2007), Etymological Dictionary of the Slavic Inherited Lexicon, Brill.
• [34]. Jacob DEVLIN, Ming-Wei CHANG, Kenton LEE, and Kristina TOUTANOVA (2019), BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding, in Jill BURSTEIN, Christy DORAN, and Thamar SOLORIO, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, Association for Computational Linguistics, Minneapolis, Minnesota, doi:10.18653/v1/N19-1423.
• [35]. Sander DIELEMAN, Jan SCHLÜTER, Colin RAFFEL, Eben OLSON, Søren Kaae SØNDERBY, Daniel NOURI, Daniel MATURANA, Martin THOMA, Eric BATTENBERG, Jack KELLY, Jeffrey De FAUW, Michael HEILMAN, Diogo Moitinho DE ALMEIDA, Brian MCFEE, Hendrik WEIDEMAN, Gábor TAKÁCS, Peter DE RIVAZ, Jon CRALL, Gregory SANDERS, Kashif RASUL, Cong LIU, Geoffrey FRENCH, and Jonas DEGRAVE (2015), Lasagne: First Release., doi:10.5281/zenodo.27878.
• [36]. Gabriel DOYLE, Klinton BICKNELL, and Roger LEVY (2014), Nonparametric Learning of Phonological Constraints in Optimality Theory, in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1094-1103, Association for Computational Linguistics, Baltimore, Maryland, doi:10.3115/v1/P14-1103.
• [37]. John DUCHI, Elad HAZAN, and Yoram SINGER (2011), Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, Journal of Machine Learning Research, 12(Jul):2121-2159.
• [38]. Michael DUNN (2012), Indo-European Lexical Cognacy Database (IELex).
• [39]. John R. FIRTH (1957), A Synopsis of Linguistic Theory, 1930-1955, Studies in linguistic analysis.
• [40]. Andrea K. FISCHER, Jilles VREEKEN, and Dietrich KLAKOW (2018), Beyond Pairwise Similarity: Quantifying and Characterizing Linguistic Similarity between Groups of Languages by MDL, Computación y Sistemas, 21(4), ISSN 2007-9737, 1405-5546, doi:10.13053/cys-21-4-2865.
• [41]. Robert FORKEL, Johann-Mattis LIST, Simon J. GREENHILL, Christoph RZYMSKI, Sebastian BANK, Michael CYSOUW, Harald HAMMARSTRÖM, Martin HASPELMATH, Gereon A. KAIPING, and Russell D. GRAY (2018), Cross-Linguistic Data Formats, Advancing Data Sharing and Re-Use in Comparative Linguistics, Scientific Data, 5:180205, ISSN 2052-4463, doi:10.1038/sdata.2018.205.
• [42]. Clémentine FOURRIER (2020), Évolution phonétique des langues et réseaux de neurones: travaux préliminaires, in Christophe BENZITOUN, Chloé BRAUD, Laurine HUBER, David LANGLOIS, Slim OUNI, and Sylvain POGODALLA, editors, Actes de la 6e conférence conjointe Journées d’Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition), volume 3: Rencontre des Étudiants Chercheurs en Informatique pour le TAL.
• [43]. Clémentine FOURRIER and Benoît SAGOT (2020a), Comparing Statistical and Neural Models for Learning Sound Correspondences, in LT4HALA 2020: First Workshop on Language Technologies for Historical and Ancient Languages, Marseille, France.
• [44]. Clémentine FOURRIER and Benoît SAGOT (2020b), Methodological Aspects of Developing and Managing an Etymological Lexical Resource: Introducing EtymDB-2.0, in Nicoletta CALZOLARI, Frédéric BÉCHET, Philippe BLACHE, Khalid CHOUKRI, Christopher CIERI, Thierry DECLERCK, Sara GOGGI, Hitoshi ISAHARA, Bente MAEGAARD, Joseph MARIANI, Hélène MAZO, Asuncion MORENO, Jan ODIJK, and Stelios PIPERIDIS, editors, Proceedings of the 12th Language Resources and Evaluation Conference, pp. 3207-3216, European Language Resources Association, Marseille, France, ISBN 979-10-95546-34-4.
• [45]. Xavier GLOROT and Yoshua BENGIO (2010), Understanding the Difficulty of Training Deep Feedforward Neural Networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249-256.
• [46]. Russell D. GRAY and Quentin D. ATKINSON (2003), Language-Tree Divergence Times Support the Anatolian Theory of Indo-European Origin, Nature, 426(6965):435-439.
• [47]. Simon J. GREENHILL, Chieh-Hsi WU, Xia HUA, Michael DUNN, Stephen C. LEVINSON, and Russell D. GRAY (2017), Evolutionary Dynamics of Language Systems, Proceedings of the National Academy of Sciences, 114(42):E8822-E8829.
• [48]. Peter GRUNWALD (2004), A Tutorial Introduction to the Minimum Description Length Principle, arXiv:math/0406077.
• [49]. Harald HAMMARSTRÖM, Robert FORKEL, Martin HASPELMATH, and Sebastian BANK (2020), Glottolog 4.2.1.
• [50]. Sepp HOCHREITER and Jürgen SCHMIDHUBER (1997), Long Short-Term Memory, Neural Computation, 9(8):1735-1780.
• [51]. Daniel J. HRUSCHKA, Simon BRANFORD, Eric D. SMITH, Jon WILKINS, Andrew MEADE, Mark PAGEL, and Tanmoy BHATTACHARYA (2015), Detecting Regular Sound Changes in Linguistics as Events of Concerted Evolution, Current Biology, 25(1):1-9.
• [52]. Jaime HUERTA-CEPAS, François SERRA, and Peer BORK (2016), ETE 3: Reconstruction, Analysis, and Visualization of Phylogenomic Data, Molecular Biology and Evolution, 33(6):1635-1638.
• [53]. Diana INKPEN, Oana FRUNZA, and Grzegorz KONDRAK (2005), Automatic Identification of Cognates and False Friends in French and English, in Proceedings of the International Conference Recent Advances in Natural Language Processing, pp. 251-257.
• [54]. Gerhard JÄGER (2014), Phylogenetic Inference from Word Lists Using Weighted Alignment with Empirically Determined Weights, in Quantifying Language Dynamics, pp. 155-204, Brill.
• [55]. Gerhard JÄGER (2015), Support for Linguistic Macrofamilies from Weighted Sequence Alignment, Proceedings of the National Academy of Sciences, 112(41):12752-12757.
• [56]. Gerhard JÄGER (2018), Global-Scale Phylogenetic Linguistic Inference from Lexical Resources, Scientific Data, 5(1):180189, ISSN 2052-4463, doi:10.1038/sdata.2018.189.
• [57]. Gerhard JÄGER (2019), Computational Historical Linguistics, Theoretical Linguistics, 45(3-4):151-182, ISSN 0301-4428, 1613-4060, doi:10.1515/tl-2019-0011.
• [58]. Gerhard JÄGER and Johann-Mattis LIST (2016), Statistical and Computational Elaborations of the Classical Comparative Method.
• [59]. Gerhard JÄGER, Johann-Mattis LIST, and Pavel SOFRONIEV (2017), Using Support Vector Machines and State-of-the-Art Algorithms for Phonetic Alignment to Identify Cognates in Multi-Lingual Wordlists, in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 1205-1216, Association for Computational Linguistics, Valencia, Spain, doi:10.18653/v1/E17-1113.
• [60]. Gerhard JÄGER and Pavel SOFRONIEV (2016), Automatic Cognate Classification with a Support Vector Machine, in Proceedings of the 13th Conference on Natural Language Processing, volume 16.
• [61]. Yoon KIM, Yacine JERNITE, David SONTAG, and Alexander M. RUSH (2016), Character-Aware Neural Language Models, in Thirtieth AAAI Conference on Artificial Intelligence.
• [62]. John D. LAFFERTY, Andrew MCCALLUM, and Fernando C. N. PEREIRA (2001), Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, in Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, pp. 282-289, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ISBN 1-55860-778-1.
• [63]. Roger LASS (1997), Historical Linguistics and Language Change, Cambridge University Press, Cambridge.
• [64]. Vladimir I. LEVENSHTEIN (1966), Binary Codes Capable of Correcting Deletions, Insertions, and Reversals, in Soviet Physics Doklady, volume 10, pp. 707-710.
• [65]. Johann-Mattis LIST (2012), LexStat: Automatic Detection of Cognates in Multilingual Wordlists, in Miriam BUTT, Sheelagh CARPENDALE, Gerald PENN, Jelena PROKIĆ, and Michael CYSOUW, editors, Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH, pp. 117-125, Association for Computational Linguistics, Avignon, France.
• [66]. Johann-Mattis LIST (2019a), Automatic Inference of Sound Correspondence Patterns across Multiple Languages, Computational Linguistics, 45(1):137-161, ISSN 0891-2017, 1530-9312, doi:10.1162/coli_a_00344.
• [67]. Johann-Mattis LIST (2019b), Beyond Edit Distances: Comparing Linguistic Reconstruction Systems, Theoretical Linguistics, 45(3-4):247-258, ISSN 0301-4428, 1613-4060, doi:10.1515/tl-2019-0016.
• [68]. Johann-Mattis LIST, Simon GREENHILL, Tiago TRESOLDI, and Robert FORKEL (2019), LingPy. A Python Library for Historical Linguistics, Jena: Max Planck Institute for the Science of Human History, doi:https://zenodo.org/badge/latestdoi/5137/lingpy/lingpy.
• [69]. Thomas MAILUND and Christian NS PEDERSEN (2004), QDist – Quartet Distance between Evolutionary Trees, Bioinformatics, 20(10):1636-1637.
• [70]. Carlo MELONI, Shauli RAVFOGEL, and Yoav GOLDBERG (2019), Ab Antiquo: Proto-Language Reconstruction with RNNs, arXiv:1908.02477 [cs].
• [71]. Tomas MIKOLOV, Ilya SUTSKEVER, Kai CHEN, Greg S. CORRADO, and Jeff DEAN (2013), Distributed Representations of Words and Phrases and Their Compositionality, in Advances in Neural Information Processing Systems, pp. 3111-3119.
• [72]. Andrea MULLONI (2007), Automatic Prediction of Cognate Orthography Using Support Vector Machines, in Chris BIEMANN, Violeta SERETAN, and Ellen RILOFF, editors, Proceedings of the ACL 2007 Student Research Workshop, pp. 25-30, Association for Computational Linguistics, Prague, Czech Republic.
• [73]. Yugo MURAWAKI (2017), Diachrony-Aware Induction of Binary Latent Representations from Typological Features, in Proceedings of the Eighth International Joint Conference on Natural Language Processing, volume 1: Long papers, pp. 451-461.
• [74]. Saul B. NEEDLEMAN and Christan D. WUNSCH (1970), A Gene Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins, Journal of Molecular Biology, 48:443-453.
• [75]. Jeffrey PENNINGTON, Richard SOCHER, and Christopher MANNING (2014), Glove: Global Vectors for Word Representation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543.
• [76]. Simone POMPEI, Vittorio LORETO, and Francesca TRIA (2011), On the Accuracy of Language Trees, PLoS One, 6(6):e20109.
• [77]. Taraka RAMA (2016), Siamese Convolutional Networks for Cognate Identification, in Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers.
• [78]. Taraka RAMA and Johann-Mattis LIST (2019), An Automated Framework for Fast Cognate Detection and Bayesian Phylogenetic Inference in Computational Historical Linguistics, in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6225-6235, Association for Computational Linguistics, Florence, Italy, doi:10.18653/v1/P19-1627.
• [79]. Sanda REINHEIMER RIPEANU (2001), Lingvistica Romanica: Lexic, Morfologie, Fonetica.
• [80]. Jorma RISSANEN (1978), Modeling by Shortest Data Description, Automatica, 14(5):465-471, ISSN 00051098, doi:10.1016/0005-1098(78)90005-5.
• [81]. Frank ROSENBLATT (1958), The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Psychological review, 65(6):386-408, doi:10.1037/h0042519.
• [82]. Naruya SAITOU and Masatoshi NEI (1987), The Neighbor-Joining Method: A New Method for Reconstructing Phylogenetic Trees, Molecular Biology and Evolution, 4(4):406-425.
• [83]. Robert. R. SOKAL and Charles. D. MICHENER (1958), A Statistical Method for Evaluating Systematic Relationships, University of Kansas Scientific Bulletin, 28:1409-1438.
• [84]. Ilya SUTSKEVER, Oriol VINYALS, and Quoc V. LE (2014), Sequence to Sequence Learning with Neural Networks, in Advances in Neural Information Processing Systems, pp. 3104-3112.
• [85]. Stijn Marinus VAN DONGEN (2000), Graph Clustering by Flow Simulation, Ph.D. thesis, University of Utrecht.
• [86]. Ashish VASWANI, Noam SHAZEER, Niki PARMAR, Jakob USZKOREIT, Llion JONES, Aidan N. GOMEZ, Łukasz KAISER, and Illia POLOSUKHIN (2017), Attention Is All You Need, in I. GUYON, U. V. LUXBURG, S. BENGIO, H. WALLACH, R. FERGUS, S. VISHWANATHAN, and R. GARNETT, editors, Advances in Neural Information Processing Systems, volume 30, pp. 5998-6008, Curran Associates, Inc.
• [87]. Andrew J. VITERBI (1967), Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm, IEEE Transactions on Information Theory, 13(2):260-269, ISSN 0018-9448, doi:10.1109/TIT.1967.1054010.
• [88]. Hannes WETTIG, Suvi HILTUNEN, and Roman YANGARBER (2011), MDL-Based Models for Alignment of Etymological Data, in Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pp. 111-117.
• [89]. Martijn WIELING, Jelena PROKIĆ, and John NERBONNE (2009), Evaluating the Pairwise String Alignment of Pronunciations, in Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education, pp. 26-34, Association for Computational Linguistics.
• [90]. Shijie WU and Ryan COTTERELL (2019), Exact Hard Monotonic Attention for Character-Level Transduction, in Preslav NAKOV and Alexis PALMER, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1530-1537, Association for Computational Linguistics, Florence, Italy, doi:10.18653/v1/P19-1148.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).