Evaluation of Neural Network Transformer Models for Named-Entity Recognition on Low-Resourced Languages

• Autorzy: Hanslo Ridewaan

Identyfikatory

DOI

10.15439/2021F7

• Konferencja: Federated Conference on Computer Science and Information Systems (16 ; 02-05.09.2021 ; online)
• Języki publikacji: EN

Abstrakty

EN

In this paper, transformer models are used to evaluate ten low-resourced South African languages for NER. Further, these transformer models are compared to bi-LSTM-aux and CRF models. The transformer models have the highest F-score of 84%. This result is significant within the context of the study, as previous research could not achieve F-scores of 80%. However, the CRF and bi-LSTM-aux models remain top performers in sequence tagging. Transformer models are viable for low-resourced languages. Future research could improve upon these findings by implementing a linear-complexity recurrent transformer variant.

Słowa kluczowe

EN

learning; artificial intelligence; natural language processing; neural nets

PL

uczenie się; sztuczna inteligencja; przetwarzanie języka naturalnego; sieci neuronowe

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2021
• Tom: Vol. 25
• Strony: 115--119
• Opis fizyczny: Bibliogr. 12 poz., wz., tab.

Twórcy

autor

Hanslo Ridewaan

• University of Pretoria Gauteng, South Africa

Bibliografia

• 1. M. Loubser, and M. J. Puttkammer, “Viability of neural networks for core technologies for resource-scarce languages”. Information, Switzerland, 2020. https://doi.org/10.3390/info11010041
• 2. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov, Unsupervised Cross-lingual Representation Learning at Scale, 2020. https://doi.org/10.18653/v1/2020.acl-main.747
• 3. B. Plank, A. Søgaard, and Y. Goldberg, “Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss”. 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers, 2016. https://doi.org/10.18653/v1/p16-2067
• 4. G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, “Neural architectures for named entity recognition”. 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference, 2016. https://doi.org/10.18653/v1/n16-1030
• 5. J. Lafferty, A. McCallum, and C. N. F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data”. ICML ’01: Proceedings of the Eighteenth International Conference on Machine Learning, 2001 https://doi.org/10.29122/mipi.v11i1.2792
• 6. E. D. Liddy, “Natural Language Processing. In Encyclopedia of Library and Information Science”. In Encyclopedia of Library and Information Science, 2001.
• 7. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need”. Advances in Neural Information Processing Systems, 2017.
• 8. M. A. Hedderich, D. Adelani, D. Zhu, J. Alabi, U. Markus, and D. Klakow, Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages, 2020. https://doi.org/10.18653/v1/2020.emnlp-main.204
• 9. R. Eiselen, “Government domain named entity recognition for South African languages”. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016, 2016.
• 10. T. Pires, E. Schlinger, and D. Garrette, “How multilingual is multilingual BERT?” ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2020. https://doi.org/10.18653/v1/p19-1493
• 11. A. Conneau, and G. Lample, “Cross-lingual language model pretraining”. Advances in Neural Information Processing Systems, 2019.
• 12. M. Sokolova, and G. Lapalme, “A systematic analysis of performance measures for classification tasks”. Information Processing and Management, 45(4), 2009. https://doi.org/10.1016/j.ipm.2009.03.002

Uwagi

1. Track 1: Artificial Intelligence in Applications

2. Session: 15th International Symposium Advances in Artificial Intelligence and Applications

3. Short Paper