Swarm algorithms for NLP : the case of limited training data

• Autorzy: Tambouratzis George , Vassiliou Marina

Identyfikatory

DOI

10.2478/jaiscr-2019-0005

• Języki publikacji: EN

Abstrakty

EN

The present article describes a novel phrasing model which can be used for segmenting sentences of unconstrained text into syntactically-defined phrases. This model is based on the notion of attraction and repulsion forces between adjacent words. Each of these forces is weighed appropriately by system parameters, the values of which are optimised via particle swarm optimisation. This approach is designed to be language-independent and is tested here for different languages. The phrasing model’s performance is assessed per se, by calculating the segmentation accuracy against a golden segmentation. Operational testing also involves integrating the model to a phrase-based Machine Translation (MT) system and measuring the translation quality when the phrasing model is used to segment input text into phrases. Experiments show that the performance of this approach is comparable to other leading segmentation methods and that it exceeds that of baseline systems.

Słowa kluczowe

EN

particle swarm optimisation; natural language processing; text phrasing; machine translation

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2019
• Tom: Vol. 9, No. 3
• Strony: 219--234
• Opis fizyczny: Bibliogr. 27 poz., rys.

Twórcy

autor

Tambouratzis George

• Institute of Language & Speech Processing, RC ”Athena” Artemidos 6 & Epidavrou str., 15125 Athens, Greece

autor

Vassiliou Marina

• Institute of Language & Speech Processing, RC ”Athena” Artemidos 6 & Epidavrou str., 15125 Athens, Greece

Bibliografia

• [1] D. Klein and C. D. Manning, A generative constituent-context model for improved grammar induction, Proceedings of 40th ACL Meeting, Philadelphia, USA, pages 128–135, July 2002.
• [2] D. Klein and C. D. Manning, Corpus-based induction of syntactic structure: Models of dependency and constituency, Proceedings of 42nd ACL Meeting, Barcelona, Spain, pages 478–485, July 21-26, 2004.
• [3] Y. Seginer, Fast unsupervised incremental parsing, Proceedings of 45th ACL Meeting, Prague, Czech Republic, pages 384–391, June 2007.
• [4] E. Ponvert, J. Baldridge, and K. Erk, Simple unsupervised grammar induction from raw text with cascaded finite state models, Proceedings of 49th ACL Meeting, Portland, Oregon, USA, pages 1077–1086, 2011.
• [5] D. Yarowsky and G. Ngai, Inducing multilingual PoS taggers and np bracketers via robust projection across aligned corpora, Proceedings of NAACL 2001 Conference, Pittsburgh, PA, USA, pages 200-207, 2-7 June 2001.
• [6] L. Zhu, D.F. Wong, and L.S. Chao, Unsupervised chunking based on graph propagation from bilingual corpus, The Scientific World Journal, Article ID 401943, 2014.
• [7] S. Goldwater, T.L. Griffiths, and M. Johnson, Contextual dependencies in unsupervised word segmentation, Proceedings of 21st International Conference on Computational Linguistics and 44th ACL Meeting, Sydney, Australia, pages 673–680, 2006.
• [8] D. Mochihashi, T. Yamada, and N. Ueda, Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling, Proceedings of 47th ACL Meeting, Suntec, Singapore, pages 100–108, August 2009.
• [9] T. Nguyen, S. Vogel, and N.A. Smith, Nonparametric word segmentation for machine translation, Proceedings of COLING-2010, Beijing, China, pages 815– 823, August 2010.
• [10] G. Tambouratzis, M. Vassiliou, & S. Sofianopoulos, Machine Translation with Minimal Reliance on Parallel Resources, Springer Briefs in Statistics series, Springer-Verlag, Berlin, 2017.
• [11] E.F. Tjong, K. Sang, and H. Dejean, Introduction to the CONLL-2001 shared task: Clause identification, Proceedings of CoNLL-2001, Toulouse,France, pages 53–57, 6-7 July 2001.
• [12] J. Lafferty, A. McCallum and F.C.N. Pereira, Conditional random fields: Probabilistic models for segmenting and labelling sequence data. ICML Conference Proceedings, Williamstown, USA, pages 282–289, 28 June-1 July 2001.
• [13] F. Sha and F.C.N. Pereira, Shallow parsing with conditional random fields, Proceedings of HLTNAACL Conference, Alberta, Canada, pages 213–220, 27 May – 1 Jun 2003.
• [14] Y. Tsuruoka, J. Tsujii, and S. Ananiadou, Fast full parsing by linear-chain conditional random fields, 12th EACL Conference Proceedings, Athens, Greece, pages 790–798, 30 March-3 April 2009.
• [15] R.O. Duda, P. E. Hart, and D.G. Stork, Pattern Classification (2nd edition), Wiley Interscience,New York, U.S.A., 2001.
• [16] G. Tambouratzis, Conditional Random Fields versus template-matching in MT phrasing tasks involving sparse training data, Pattern Recognition Letters, 53:44-52, 2015.
• [17] A.J. Booker, J.E. Dennis Jr., P.D. Frank, D.B. Serafini, V. Torczon, and M.W. Trosset, A Rigorous Framework for Optimization of Expensive Functions by Surrogates, Structural Optimisation, 17:1-13, 1999.
• [18] Y. Jin, Surrogate-assisted evolutionary computation: Recent advances and future challenges, Swarm and Evolutionary Computation, 1(2):61-70, 2011.
• [19] K. Stoeber, P. Wagner, J. Helbig, S. Koester, D. Stall, M. Thomae, J. Blauert, W. Hess, and H. Mangold, Speech synthesis by multilevel selection and concatenation of units from large speech corpora. Verbmobil: Foundations of Speech-to-Speech Translation, Symbolic Computation, Springer, Berlin, pages 519–537, 2000.
• [20] J. Kennedy and R. Eberhart, Particle swarm optimization, IEEE International Conference on Neural Networks, Perth, WA, USA, pages 1942–1948, 27 Nov-1 Dec 1995.
• [21] G. Tambouratzis, Applying PSO to Natural Language Processing Tasks: Optimizing the Identification of Syntactic Phrases, Proceedings of CEC-2016 Conference, Vancouver, Canada, pp. 1831-1838, July 2016.
• [22] R. Mendes, J. Kennedy, and J. Neves, The fully informed particle swarm: Simpler maybe better, IEEE Transactions on Evolutionary Computation, 8(3):204–210, 2004.
• [23] Z.-H. Zhan, J. Zhang, Y. Li, and H.S.-H. Chung, Adaptive particle swarm optimization, IEEE Transactions on Systems, Man Cybernetics Part B: Cybernetics, 39(6):1362–1381, 2009.
• [24] G. Tambouratzis (2017) The effectiveness of surrogate functions in improving the accuracy of PSOtype algorithms in an NLP task, Proceedings of 10th SSCI Conference, Honolulu, USA, pp. 3214-3221,IEEE Press, 27 November - 1 December 2017.
• [25] K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. 40th ACL Meeting Proceedings, Philadelphia, USA, pages 311– 318.
• [26] NIST. Automatic evaluation of machine translation quality using n-gram cooccurrences statistics. Report available at http://www.itl.nist.gov/iad/mig/tests/mt/doc/ngramstudy.pdf, 2002.
• [27] K.R. Harrison., A.P. Engelbrecht, B.M. Ombuki-Berman, An adaptive particle swarm optimization algorithm based on optimal parameter regions, Proceedings of 10th SSCI Conference, Honolulu, USA, pp. 1606-1613, IEEE Press, 27 November - 1 December 2017.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).