Interfacing language, spatial perception and cognition in Type Theory with Records

• Autorzy: Dobnik S. , Cooper R.

Identyfikatory

DOI

10.15398/jlm.v5i2.148

• Języki publikacji: EN

Abstrakty

EN

We argue that computational modelling of perception, action, language, and cognition introduces several requirements of a formal semantic theory and its practical implementations in situated dialogue agents. Using examples of semantic representations of spatial descriptions we show how Type Theory with Records (TTR) satisfies these requirements and provides a promising knowledge representation system for situated agents.

Słowa kluczowe

EN

spatial language; TTR; Type Theory with Records; computational framework

• Wydawca: Instytut Podstaw Informatyki PAN
• Czasopismo: Journal of Language Modelling
• Rocznik: 2017
• Tom: Vol. 5, No. 2
• Strony: 273--301
• Opis fizyczny: Bibliogr. 63 poz., rys.

Twórcy

autor

Dobnik S.

• Department of Philosophy, Linguistics & Theory of Science (FLOV), Centre for Linguistic Theory and Studies in Probability (CLASP), University of Gothenburg, Sweden

autor

Cooper R.

• Department of Philosophy, Linguistics & Theory of Science (FLOV), Centre for Linguistic Theory and Studies in Probability (CLASP), University of Gothenburg, Sweden

Bibliografia

• [1] James F Allen (1983), Maintaining knowledge about temporal intervals, Communications of the ACM, 26 (11): 832-843.
• [2] John A. Bateman, Joana Hois, Robert Ross, and Thora Tenbrink (2010), A linguistic ontology of space for natural language processing, Artificial Intelligence, 174 (14): 1027-1071.
• [3] Steven Bird, Ewan Klein, and Edward Loper (2009), Natural language processing with Python, O’Reilly, http://nltk.org/book/.
• [4] Patrick Blackburn and Johan Bos (2005), Representation and inference for natural language. A first course in computational semantics, CSLI Publications.
• [5] Anthony G. Cohn and Jochen Renz (2008), Qualitative Spatial Representation and Reasoning, in Vladimir Lifschitz Frank van Harmelen and Bruce Porter, editors, Handbook of Knowledge Representation, volume 3 of Foundations of Artificial Intelligence, chapter 13, pp. 551-596, Elsevier.
• [6] Robin Cooper (2012), Type theory and semantics in flux, in Ruth Kempson, Nicholas Asher, and Tim Fernando, editors, Handbook of the Philosophy of Science, volume 14 of General editors: Dov M Gabbay, Paul Thagard and John Woods, Elsevier BV.
• [7] Robin Cooper (2017), Adapting Type Theory with Records for Natural Language Semantics, in Stergios Chatzikyriakidis and Zhaohui Luo, editors, Modern Perspectives in Type-Theoretical Semantics, number 98 in Studies in Linguistics and Philosophy, pp. 71-94, Springer.
• [8] Robin Cooper (in prep), Type theory and language: from perception to linguistic communication, https://sites.google.com/site/typetheorywithrecords/drafts, draft of book chapters.
• [9] Robin Cooper, Simon Dobnik, Shalom Lappin, and Staffan Larsson (2015), Probabilistic Type Theory and Natural Language Semantics, Linguistic Issues in Language Technology — LiLT, 10 (4): 1-43.
• [10] Fintan J. Costello and John D. Kelleher (2006), Spatial prepositions in context: the semantics of near in the presence of distractor objects, in Proceedings of the Third ACL-SIGSEM Workshop on Prepositions, Prepositions ’06, pp. 1-8, Association for Computational Linguistics, Stroudsburg, PA, USA.
• [11] Kenny R. Coventry, Angelo Cangelosi, Rohanna Rajapakse, Alison Bacon, Stephen Newstead, Dan Joyce, and Lynn V. Richards (2005), Spatial Prepositions and Vague Quantifiers: Implementing the Functional Geometric Framework, in Christian Freksa, Markus Knauff, Bernd Krieg-Brückner, Bernhard Nebel, and Thomas Barkowsky, editors, Spatial Cognition IV. Reasoning, Action, Interaction, volume 3343 of Lecture Notes in Computer Science, pp. 98-110, Springer Berlin Heidelberg.
• [12] Kenny R. Coventry and Simon C. Garrod (2005), Spatial prepositions and the functional geometric framework. Towards a classification of extra-geometric influences, Functional features in language and space: Insights from perception, categorisation and development, pp. 163-173.
• [13] Kenny R. Coventry, Mercè Prat-Sala, and Lynn Richards (2001), The interplay between geometry and function in the apprehension of Over, Under, Above and Below, Journal of Memory and Language, 44 (3): 376-398.
• [14] M. W. M. G Dissanayake, P. M. Newman, H. F. Durrant-Whyte, S. Clark, and M. Csorba (2001), A solution to the simultaneous localization and map building (SLAM) problem, IEEE Transactions on Robotic and Automation, 17 (3): 229-241.
• [15] Simon Dobnik (2009), Teaching mobile robots to use spatial words, Ph.D. thesis, University of Oxford: Faculty of Linguistics, Philology and Phonetics and The Queen’s College, Oxford, United Kingdom, http://www.dobnik.net/simon/documents/thesis.pdf.
• [16] Simon Dobnik and Amelie Åstbom (2017), (Perceptual) grounding as interaction, in Volha Petukhova and Ye Tian, editors, Proceedings of Saardial – Semdial 2017: The 21st Workshop on the Semantics and Pragmatics of Dialogue, pp. 1-9, Saarbrücken, Germany.
• [17] Simon Dobnik, Robin Cooper, and Staffan Larsson (2013), Modelling Language, Action, and Perception in Type Theory with Records, in Denys Duchier and Yannick Parmentier, editors, Constraint Solving and Language Processing: 7th International Workshop, CSLP 2012, Orléans, France, September 13-14, 2012, Revised Selected Papers, volume 8114 of Lecture Notes in Computer Science, pp. 70-91, Springer Berlin Heidelberg.
• [18] Simon Dobnik and Erik de Graaf (2017), KILLE: a Framework for Situated Agents for Learning Language Through Interaction, in Jörg Tiedemann, editor, Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa), volume 131 of Linköping Electronic Conference Proceedings and NEALT Proceedings Series Vol. 29, pp. 1-10, Northern European Association for Language Technology (NEALT), Linköping University Electronic Press, Gothenburg, Sweden.
• [19] Simon Dobnik, Christine Howes, and John D. Kelleher (2015), Changing perspective: Local alignment of reference frames in dialogue, in Christine Howes and Staffan Larsson, editors, Proceedings of goDIAL – Semdial 2015: The 19th Workshop on the Semantics and Pragmatics of Dialogue, pp. 24-32, Gothenburg, Sweden.
• [20] Simon Dobnik and John D. Kelleher (2013), Towards an automatic identification of functional and geometric spatial prepositions, in Proceedings of PRE-CogSsci 2013: Production of referring expressions – bridging the gap between cognitive and computational approaches to reference, pp. 1-6, Berlin, Germany.
• [21] Simon Dobnik and John D. Kelleher (2014), Exploration of functional semantics of prepositions from corpora of descriptions of visual scenes, in Proceedings of the Third V&L Net Workshop on Vision and Language, pp. 33-37, Dublin City University and the Association for Computational Linguistics, Dublin, Ireland.
• [22] Simon Dobnik and John D. Kelleher (2016), A Model for Attention-Driven Judgements in Type Theory with Records, in Julie Hunter, Mandy Simons, and Matthew Stone, editors, JerSem: The 20th Workshop on the Semantics and Pragmatics of Dialogue, volume 20, pp. 25-34, New Brunswick, NJ USA.
• [23] Simon Dobnik, John D. Kelleher, and Christos Koniaris (2014), Priming and Alignment of Frame of Reference in Situated Conversation, in Verena Rieser and Philippe Muller, editors, Proceedings of DialWatt – Semdial 2014: The 18th Workshop on the Semantics and Pragmatics of Dialogue, pp. 43-52, Edinburgh.
• [24] David R Dowty, Robert Eugene Wall, and Stanley Peters (1981), Introduction to Montague semantics, D. Reidel Pub. Co., Dordrecht, Holland.
• [25] Zachary Estes, Sabrina Golonka, and Lara L Jones (2011), Thematic Thinking: The Apprehension and Consequences of Thematic Relations, in Brian Ross, editor, The Psychology of Learning and Motivation, volume 54, pp. 249-294, Burlington: Academic Press.
• [26] Ronald Fagin, Joseph Y. Halpern, Yoram Moses, and Moshe Y. Vardi (1995), Reasoning about knowledge, MIT Press, Cambridge, Mass.
• [27] Christiane Fellbaum (1998), WordNet: an electronic lexical database, MIT Press, Cambridge, Mass.
• [28] Gottlob Frege (1948), Sense and Reference, The Philosophical Review, 57 (3): 209-230.
• [29] Klaus-Peter Gapp (1994a), Basic Meanings of Spatial Relations: Computation and Evaluation in 3D Space, in Barbara Hayes-Roth and Richard E. Korf, editors, AAAI, pp. 1393-1398, AAAI Press/The MIT Press.
• [30] Klaus-Peter Gapp (1994b), A computational model of the basic meanings of graded composite spatial relations in 3D space, in Advanced geographic data modelling. Spatial data modelling and query languages for 2D and 3D applications (Proceedings of the AGDM’94), Publications on Geodesy 40, pp. 66-79, Netherlands Geodetic Commission.
• [31] Simon Garrod and Gwyneth Doherty (1994), Conversation, co-ordination and convention: An empirical investigation of how groups establish linguistic conventions, Cognition, 53 (3): 181-215.
• [32] Stevan Harnad (1990), The symbol grounding problem, Physica D, 42 (1-3): 335-346.
• [33] Annette Herskovits (1986), Language and spatial cognition: an interdisciplinary study of the prepositions in English, Cambridge University Press, Cambridge.
• [34] Serena Ivaldi, Sao Mai Nguyen, Natalia Lyubova, Alain Droniou, Vincent Padois, David Filliat, Pierre-Yves Oudeyer, and Sigaud Olivier (2014), Object Learning Through Active Exploration, IEEE Transactions on Autonomous Mental Development, 6 (1): 56-72.
• [35] John D. Kelleher, Fintan J. Costello, and Josef van Genabith (2005), Dynamically Structuring Updating and Interrelating Representations of Visual and Linguistic Discourse, Artificial Intelligence, 167: 62-102.
• [36] D. Kelleher, Robert J. Ross, Colm Sloan, and Brian Mac Namee (John 2011), The effect of occlusion on the semantics of projective spatial terms: a case study in grounding language in perception, Cognitive Processing, 12 (1): 95-108.
• [37] Geert-Jan M. Kruijff, Hendrik Zender, Patric Jensfelt, and Henrik I. Christensen (2007), Situated dialogue and spatial organization: what, where... and why?, International Journal of Advanced Robotic Systems, 4 (1): 125-138, special issue on human and robot interactive communication.
• [38] Lars Kunze, Chris Burbridge, and Nick Hawes (2014), Bootstrapping Probabilistic Models of Qualitative Spatial Relations for Active Visual Object Search, in AAAI Spring Symposium 2014 on Qualitative Representations for Robots, Stanford University in Palo Alto, California, US.
• [39] Shalom Lappin (2013), Intensions as Computable Functions, Linguistic Issues in Language Technology, 9: 1-12.
• [40] Staffan Larsson (2015), Formal semantics for perceptual classification, Journal of Logic and Computation, 25 (2): 335-369.
• [41] Daniel Lassiter (2011), Vagueness as probabilistic linguistic knowledge, in Proceedings of the international conference on vagueness in Communications (ViC’09), pp. 127-150, Springer-Verlag, Berlin, Heidelberg.
• [42] Stanislao Lauria, Guido Bugmann, Theocharis Kyriacou, and Ewan Klein (2002), Mobile robot programming using natural language, Robotics and Autonomous Systems, 38 (3-4): 171-181.
• [43] Emilie L. Lin and Gregory L. Murphy (2001), Thematic relations in adults’ concepts, Journal of experimental psychology: General, 130 (1): 3-28.
• [44] Gordon D. Logan and Daniel D. Sadler (1996), A computational analysis of the apprehension of spatial relations, in Paul Bloom, Mary A. Peterson, Lynn Nadel, and Merrill F. Garrett, editors, Language and Space, pp. 493-530, MIT Press, Cambridge, MA.
• [45] Didier Maillat (2003), The semantics and pragmatics of directionals: a case study in English and French, Ph.D. thesis, University of Oxford: Committee for Comparative Philology and General Linguistics, Oxford, United Kingdom.
• [46] Per Martin-Löf (1984), Intuitionistic Type Theory, Bibliopolis, Naples.
• [47] Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo, and Dieter Fox (2012a), A joint model of language and perception for grounded attribute learning, in John Langford and Joelle Pineau, editors, Proceedings of the 29th International Conference on Machine Learning (ICML 2012), Edinburgh, Scotland.
• [48] Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox (2012b), Learning to Parse Natural Language Commands to a Robot Control System, in Proceedings of the 13th International Symposium on Experimental Robotics (ISER).
• [49] George A. Miller and Philip N. Johnson-Laird (1976), Language and perception, Cambridge University Press, Cambridge.
• [50] Richard Montague (1974), Formal Philosophy: Selected Papers of Richard Montague, Yale University Press, New Haven, ed. and with an introduction by Richmond H. Thomason.
• [51] Bengt Nordström, Kent Petersson, and Jan M. Smith (1990), Programming in Martin-Löf’s Type Theory, volume 7 of International Series of Monographs on Computer Science, Clarendon Press, Oxford.
• [52] Martin J. Pickering and Simon Garrod (2004), Toward a mechanistic psychology of dialogue, Behavioral and Brain Sciences, 27 (2): 169-190.
• [53] Matthew Purver, Jonathan Ginzburg, and Patrick Healey (2003), On the means for clarification in dialogue, in Current and new directions in discourse and dialogue, pp. 235-255, Springer.
• [54] Terry Regier and Laura A. Carlson (2001), Grounding spatial language in perception: an empirical and computational investigation, Journal of Experimental Psychology: General, 130 (2): 273-298.
• [55] Deb Roy (2002), Learning visually-grounded words and syntax for a scene description task, Computer speech and language, 16 (3): 353-385.
• [56] Deb Roy (2005), Semiotic schemas: a framework for grounding language in action and perception, Artificial Intelligence, 167 (1-2): 170-205.
• [57] Kristoffer Sjöö (2011), Functional understanding of space: Representing spatial knowledge using concepts grounded in an agent’s purpose, Ph.D. thesis, KTH, Computer Vision and Active Perception (CVAP), Centre for Autonomous Systems (CAS), Stockholm, Sweden.
• [58] Danijel Skočaj, Matej Kristan, Alen Vrečko, Marko Mahnič, Miroslav Janíček, Geert-Jan M. Kruijff, Marc Hanheide, Nick Hawes, Thomas Keller, Michael Zillich, and Kai Zhou (2011), A system for interactive learning in dialogue with a tutor, in IEEE/RSJ International Conference on Intelligent Robots and Systems IROS 2011, San Francisco, CA, USA.
• [59] Luc Steels and Tony Belpaeme (2005), Coordinating Perceptually Grounded Categories Through Language: A Case Study For Colour, Behavioral and Brain Sciences, 28 (4): 469-489.
• [60] Luc Steels and Martin Loetzsch (2009), Perspective Alignment in Spatial Language, in Kenny R. Coventry, Thora Tenbrink, and John. A. Bateman, editors, Spatial Language and Dialogue, Oxford University Press.
• [61] Mark Tutton (2013), A new approach to analysing static locative expressions, Language and Cognition, 5: 25-60.
• [62] Matthew E. Watson, Martin J. Pickering, and Holly P. Branigan (2004), Alignment of reference frames in dialogue, in Proceedings of the 26th Annual Conference of the Cognitive Science Society, Chicago, USA.
• [63] Joost Zwarts and Yoad Winter (2000), Vector Space Semantics: A Model-Theoretic Analysis of Locative Prepositions, Journal of Logic, Language and Information, 9: 169-211.