Using particle swarm optimization to accurately identify syntactic phrases in free text

• Autorzy: Tambouratzis G.

Identyfikatory

DOI

10.1515/jaiscr-2018-0004

• Języki publikacji: EN

Abstrakty

EN

The present article reviews the application of Particle Swarm Optimization (PSO) algorithms to optimize a phrasing model, which splits any text into linguistically-motivated phrases. In terms of its functionality, this phrasing model is equivalent to a shallow parser. The phrasing model combines attractive and repulsive forces between neighbouring words in a sentence to determine which segmentation points are required. The extrapolation of phrases in the specific application is aimed towards the automatic translation of unconstrained text from a source language to a target language via a phrase-based system, and thus the phrasing needs to be accurate and consistent to the training data. Experimental results indicate that PSO is effective in optimising the weights of the proposed parser system, using two different variants, namely sPSO and AdPSO. These variants result in statistically significant improvements over earlier phrasing results. An analysis of the experimental results leads to a proposed modification in the PSO algorithm, to prevent the swarm from stagnation, by improving the handling of the velocity component of particles. This modification results in more effective training sequences where the search for new solutions is extended in comparison to the basic PSO algorithm. As a consequence, further improvements are achieved in the accuracy of the phrasing module.

Słowa kluczowe

EN

parsing of natural language; machine translation; syntactically-derived phrasing; particle swarm optimization (PSO); PSO; parameter optimization; adaptive PSO; AdPSO

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2018
• Tom: Vol. 8, No. 1
• Strony: 63--77
• Opis fizyczny: Bibliogr. 35 poz., rys.

Twórcy

autor

Tambouratzis G.

• Dept. of Machine Translation, Institute for Language and Speech Processing / Athena Research Centre 6 Artemidos & Epidavrou Str., Paradissos Amaroussiou, 151 25, Greece

Bibliografia

• [1] Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, Robert L. Mercer, The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, Vol. 19, No.2, pp. 263-311, 1993.
• [2] Philipp Koehn, Statistical Machine Translation. Cambridge University Press, Cambridge, 2010.
• [3] Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio, On the properties of neural machine translation: Encoder–decoder approaches. Proceedings of Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 2014.
• [4] George Tambouratzis, Sokratis Sofianopoulos Marina Vassiliou, Language-independent hybrid MT with PRESEMT. Proceedings of 2nd HYTRAWorkshop, held within the ACL-2013 Conference, Sofia, Bulgaria, pp. 123-130, 2013.
• [5] James Kennedy, Russel C. Eberhart, Particle Swarm Optimization. Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, November, pp. 1942-1947, 1995.
• [6] John Lafferty, Andrew McCallum, Fernando C.N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labelling Sequence Data. Proceedings of ICML Conference, Williamstown, USA, pp. 282-289, 2001.
• [7] Fei Sha, Fernando C.N. Pereira, Shallow Parsing with Conditional Random Fields. Proceedings of HLT-NAACL Conference, pp. 213-220, 2003.
• [8] J.R. Finkel, A. Kleeman and C.D. Manning, Efficient, Feature-Based, Conditional Random Field Parsing. Proceedings of ACL-2008 Meeting, Columbus, Ohio, USA, pp. 959-967, 2008.
• [9] Yoshimasa Tsuruoka, J. Tsujii, Sophia Ananiadou, Fast Full Parsing by Linear-Chain Conditional Random Fields. Proceedings of the 12th EACL Conference, Athens, Greece, pp. 790–798, 2009.
• [10] Greg Durrett, Dan Klein, Neural CRF Parsing. Proceedings of the 53rd ACL Meeting, Beijing, China, pp. 302-312, 2015.
• [11] George Tambouratzis, Conditional Random Fields versus template-matching in MT phrasing tasks involving sparse training data. Pattern Recognition Letters, Vol. 53, pp. 44-52, 2015.
• [12] Elva J.H. Robinson, Francis L.W. Ratnieks, M. Holcombe, An agent-based model to investigate the roles of attractive and repellent pheromones in ant decision making during foraging. Journal of Theoretical Biology, Vol. 255, pp. 250-258, 2008.
• [13] Changsheng Zhang, Jigui Sun, An Alternate two phases particle swarm optimization algorithm for flow shop scheduling problem. Expert Systems with Applications, Vol. 36, pp. 5162-5167, 2009.
• [14] S.X. Yu, J. Shi, Segmentation with Pairwise Attraction and Repulsion. Proceedings of ICCV-2001 Conference, Vancouver, Canada, pp. 52–58, 2001.
• [15] Karlheinz Stber, Petra Wagner, David Stall, Jens Helbig, Matthias Thomae, Jens Blauert, Wolfgang Hess, H. Mangold, Speech Synthesis by Multilevel Selection and Concatenation of Units from Large Speech Corpora. Verbmobil: Foundations of Speech-to-Speech Translation, Symbolic Computation, Springer, Berlin, pp. 519–537, 2000.
• [16] George Tambouratzis, Applying PSO to Natural Language Processing Tasks: Optimizing the Identification f Syntactic Phrases. Proceedings of CEC-2016 Conference, Vancouver, Canada, pp. 1831-1838, 2016.
• [17] Rui Mendes, James Kennedy, Jose Neves, The Fully Informed Particle Swarm: Simpler, Maybe Better. IEEE Transactions on Evolutionary Computation, 8 (3), 204-210, 2004.
• [18] Moayed Daneshyari, Gary G. Yen, Cultural-Based Multiobjective Particle Swarm Optimization. IEEE ransactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 41, No. 2, pp.553-567, 2011.
• [19] Xin Chen, Yangmin Li, A Modified PSO Structure Resulting in High Exploration Ability with Convergence Guaranteed. IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 37, No. 5, pp. 1271-1289, 2007.
• [20] Jens Gimmler, Thomas Sttzle, Thomas E. Exner, Hybrid Particle Swarm Optimization: An examination of the influence of iterative improvement algorithms on its behavior. Proceedings of ANTS-2006 Workshop; Brussels, Belgium, 4-7 September, Lecture Notes in Computer Science, Vol. 4150, pp. 436-443, Springer, 2006.
• [21] Richard P. Brent, Algorithms for Minimization without Derivatives, Englewood Cliffs, NJ, Prentice-Hall, Chapter 5, 1973.
• [22] William H. Press, Saul A. Teukolsky, W.T. Vetterling, Brian P. Flannery, Numerical Recipes in Fortran 77: The Art of Scientific Computing. New York, U.S.A., Cambridge University Press, Chapter 7, 2002.
• [23] James Kennedy, Rui Mendes, Neighborhood Topologies in Fully Informed and Best-of-Neighborhood Particle Swarms. IEEE Transactions on Systems, Man & Cybernetics – Part C: Applications and Reviews, Vol. 36, No. 4, pp. 515-519, 1996.
• [24] Zhi-Hui Zhan, Jun Zhang, Yun Li, Henry Shu-Hung Chung, Adaptive Particle Swarm Optimization.IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 39, No.6, pp. 1362-1381, 2009.
• [25] Helmut Schmid, Probabilistic part-of-speech tagging using decision trees. Proceedings of International Conference on New Methods in Language Processing, Manchester, UK. 1994.
• [26] Helmut Schmid, Improvements in part-of speech tagging with an application to german. Proceedings of the ACL SIGDAT-Workshop. Dublin, Ireland, Association for Computing Machinery, 1995.
• [27] Kishore Papineni, Salim Roukos, ToddWard,Wei-Jing Zhu, BLEU: A Method for Automatic Evaluation of Machine Translation. 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA, pp. 311-318, 2002.
• [28] NIST, 2002. Automatic Evaluation of Machine Translation Quality Using n-gram Co-occurrences Statistics.
• [29] Yuhui Shi, Russel C. Eberhart, Parameter selection in particle swarm optimization, In: Proceedings of the 7th International Conference on Evolutionary Programming, New York, U.S.A., pp. 591-600, 1998.
• [30] Maurice Clerc, James Kennedy, The particle swarm - explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58-73, 2002.
• [31] Jos Garca-Nieto, Enrique Alba, Restart particle swarm optimization with velocity modulation: a scalability test. Soft Computing, Vol.15, pp. 2221-2232, 2010.
• [32] George I. Evers, Mounir Ben Ghalia, Regrouping particle swarm optimization: A new global optimization algorithm with improved performance consistency across benchmarks. IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Texas, 11-14 October, pp. 3901-3908,2009.
• [33] Sabine Helwig, Frank Neumann, Rolf Wanka, Particle swarm optimization with velocity adaptation.International Conference on Adaptive and Intelligent Systems (ICAIS’09), 24 September, pp. 146-151, 2009.
• [34] Xin Chen, Yangmin Li, A Modified PSO Structure Resulting in High Exploration Ability with Convergence Guaranteed. IEEE Transactions on Systems, Man & Cybernetics – Part B: Cybernetics, Vol. 37,No. 5, pp. 1271-1289, 2007.
• [35] Lili Liu, Shengxiang Yang, Dingway Wang, Particle Swarm Optimization with Composite Particles in Dynamic Environments. IEEE Transactions on Systems, Man, And Cybernetics—Part B: Cybernetics, Vol. 40, No. 6, pp. 1634-1648, 2010.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).