QRGS : Question Responses Generation via crowdsourcing

• Autorzy: Łupkowski, Paweł , Ginzburg, Jonathan , Chmurska, Ewelina , Płatosz, Adrianna , Kwiecień, Aleksandra , Adamska, Barbara , Szkalej, Magdalena
• Języki publikacji: EN

Abstrakty

EN

QRGS stands for the Question Responses Generation System. It is an online game-like framework designed for gathering various types of question responses. A QRGS user is asked to read a simple story and impersonate its main character. As the story unfolds the user is con-fronted with four questions and (s)he is expected to answer these in the way the main character would. In this way, we obtain responses to questions of a desired type. The data gathered via QRGS is a useful supplement to the linguistic data already present in language corpora – especially for languages for which such resources are sparse. As such, it opens the possibility for better understanding of the use of questions in natural language dialogues and analysing the response space of such questions. In this paper, we present the main idea of QRGS and the re-sults of five studies (in Polish and in English) that test the framework. Our discussion addresses issues concerning the efficiency and accu-racy of the proposed approach. We also discuss the availability of the QRGS and its potential future improvements.

Słowa kluczowe

EN

gamification; crowdsourcing; questions; responses; language resource

• Czasopismo: Journal of Language Modelling
• Rocznik: 2024
• Tom: Vol. 12, No. 1
• Strony: 213--270
• Opis fizyczny: Bibliobr. 31 poz., rys. tab.

Twórcy

autor

Łupkowski, Paweł

• Adam Mickiewicz University, Poznań,

autor

Ginzburg, Jonathan

• Université Paris Cité, CNRS, Laboratoire de Linguistique Formelle, paris,

autor

Chmurska, Ewelina

• Adam Mickiewicz University, Poznań

autor

Płatosz, Adrianna

• Adam Mickiewicz University, Poznań

autor

Kwiecień, Aleksandra

• Adam Mickiewicz University, Poznań

autor

Adamska, Barbara

• Adam Mickiewicz University, Poznań

autor

Szkalej, Magdalena

• Adam Mickiewicz University, Poznań

Bibliografia

• 1. Anne H. ANDERSON, Miles BADER, Ellen Gurman BARD, Elizabeth H. BOYLE, Gwyneth M. DOHERTY, Simon C. GARROD, Stephen D. ISARD, Jacqueline C.
• 2. KOWTKO, Jan M. MCALLISTER, Jim MILLER, Catherine F. SOTILLO, Henry S.
• 3. THOMPSON, and Regina WEINERT (1991), The HCRC Map Task Corpus, Language and Speech, 34(4):351-366.
• 4. Lou BURNARD, editor (2007), Reference guide for the British National Corpus (XML Edition), Oxford University Computing Services on behalf of the BNC
• 5. Consortium, http://www.natcorp.ox.ac.uk/XMLedition/URG/, access 20.03.2017.
• 6. Jon CHAMBERLAIN, Massimo POESIO, and Udo KRUSCHWITZ (2008), PhraseDetectives: A web-based collaborative annotation game, in Proceedings of the International Conference on Semantic Systems (I-Semantics’ 08), pp. 42-49.
• 7. Seth COOPER, Adrien TREUILLE, Janos BARBERO, Andrew LEAVER-FAY, Kathleen TUITE, Firas KHATIB, Alex Cho SNYDER, Michael BEENEN, David SALESIN, David BAKER, Zoran POPOVIĆ, and >57,000 Foldit PLAYERS (2010), The challenge of designing scientific discovery games, in Proceedings of the Fifth International Conference on the Foundations of Digital Games, pp. 40-47.
• 8. Cristian DANESCU-NICULESCU-MIZIL and Lillian LEE (2011), Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs, in Proceedings of the 2nd workshop on cognitive modeling and computational linguistics, pp. 76-87, Association for Computational Linguistics.
• 9. Lorna DSILVA, Shubhi MITTAL, Brian KOEPNICK, Jeff FLATTEN, Seth COOPER, and Scott HOROWITZ (2019), Creating custom foldit puzzles for teaching biochemistry, Biochemistry and Molecular Biology Education, 47(2):133-139.
• 10. Dagmara DZIEDZIC (2016), Use of the free to play model in games with a purpose: the RoboCorp game case study, Bio-Algorithms and Med-Systems, 12(4):187-197.
• 11. Jonathan GINZBURG, Zulipiye YUSUPUJIANG, Chuyuan LI, Kexin REN, Aleksandra KUCHARSKA, and Pawel LUPKOWSKI (2022), Characterizing the response space of questions: data and theory, Dialogue & Discourse, 13(2):79-132.
• 12. Jonathan GINZBURG, Zulipiye YUSUPUJIANG, Chuyuan LI, Kexin REN, and Paweł ŁUPKOWSKI (2019), Characterizing the response space of questions: a corpus study for English and Polish, in Proceedings of the 20th annual SIGdial meeting on discourse and dialogue, pp. 320-330.
• 13. Oliwia IGNASZAK and Paweł ŁUPKOWSKI (2017), Inferential Erotetic Logic in modelling of cooperative problem solving involving questions in the QuestGen game, Organon F, 24(2):214-244.
• 14. Charlene JENNETT, Anna L. COX, Paul CAIRNS, Samira DHOPAREE, Andrew EPPS, Tim TIJS, and Alison WALTON (2008), Measuring and defining the experience of immersion in games, International Journal of Human-Computer Studies, 66(9):641-661.
• 15. Chris J. LINTOTT, Kevin SCHAWINSKI, Anže SLOSAR, Kate LAND, Steven BAMFORD, Daniel THOMAS, M. Jordan RADDICK, Robert C. NICHOL, Alex SZALAY, Dan ANDREESCU, Phil MURRAY, and Jan VANDENBERG (2008), Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey, Monthly Notices of the Royal Astronomical Society, 389(3):1179-1189.
• 16. Paweł ŁUPKOWSKI and Dagmara DZIEDZIC (2016), Building players’ engagement – a case study of games with a purpose in science, Homo Ludens, 1(9):127-145.
• 17. Paweł ŁUPKOWSKI, Mariusz URBAŃSKI, Andrzej WIŚNIEWSKI, Wojciech BŁĄDEK, Agata JUSKA, Anna KOSTRZEWA, Dominika PANKOW, Katarzyna PALUSZKIEWICZ, Oliwia IGNASZAK, Joanna URBAŃSKA, Natalia ŻYLUK, Andrzej GAJDA, and Bartosz MARCINIAK (2017), Erotetic Reasoning Corpus. A data set for research on natural question processing, Journal of Language Modelling, 5(3):607-631.
• 18. Paweł ŁUPKOWSKI and Patrycja WIETRZYCKA (2015), Gamification for question processing research – the QuestGen game, Homo Ludens, 7(1):161-171.
• 19. Adam PRZEPIÓRKOWSKI, Mirosław BAŃKO, Rafał L. GÓRSKI, Barbara LEWANDOWSKA-TOMASZCZYK, Marek ŁAZIŃSKI, and Piotr PĘZIK (2011), National Corpus of Polish, in Proceedings of the 5th language & technology conference: Human language technologies as a challenge for computer science and linguistics, pp. 259-263, Fundacja Uniwersytetu im. Adama Mickiewicza Poznań.
• 20. Piotr PĘZIK (2014), Spokes search engine for Polish conversational data, http://hdl.handle.net/11321/47, CLARIN-PL digital repository.
• 21. Carolyn P. ROSÉ, Barbara Di EUGENIO, and Johanna D. MOORE (1999), A dialogue-based tutoring system for basic electricity and electronics, in Susanne P. LAJOIE and Martial VIVET, editors, Artificial intelligence in education, pp. 759-761, IOS, Amsterdam.
• 22. Paweł STROJNY and Agnieszka STROJNY (2014), Kwestionariusz immersji – polska adaptacja i empiryczna weryfikacja narzędzia, Homo Ludens, 1(6):171-186.
• 23. Mariusz URBAŃSKI, Katarzyna PALUSZKIEWICZ, and Joanna URBAŃSKA (2016a), Erotetic problem solving: From real data to formal models. An analysis of solutions to erotetic reasoning test task, in F. PAGLIERI, L. BONETTI, and S. FELLETTI, editors, The Psychology of Argument: Cognitive Approaches to Argumentation and Persuasion, pp. 33-46, College Publications.
• 24. Mariusz URBAŃSKI, Natalia ŻYLUK, Katarzyna PALUSZKIEWICZ, and Joanna URBAŃSKA (2016b), A formal model of erotetic reasoning in solving somewhat ill-defined problems, in D. MOHAMMED and M. LEWIŃSKI, editors, Argumentation and Reasoned Action. Proceedings of the 1st European Conference on Argumentation. London: College Publications, pp. 973-983, College Publications.
• 25. Noortje VENHUIZEN, Valerio BASILE, Kilian EVANG, and Johan BOS (2013), Gamification for word sense labeling, in Proceedings of the 10th International Conference on Computational Semantics (IWCS’13) – Short Papers, pp. 397-403.
• 26. Anthony J. VIERA and Joanne M. GARRETT (2005), Understanding interobserver agreement: the kappa statistic, Family Medicine, 37(5):360-363.
• 27. Aleksandra WASIELEWSKA and Paweł ŁUPKOWSKI (2022), IMUW the questionnaire measuring the engagement of attention in a task execusion, https://osf.io/6dt8f/.
• 28. Andrzej WIŚNIEWSKI (2013), Questions, inferences and scenarios, College Publications, London.
• 29. Zulipiye YUSUPUJIANG and Jonathan GINZBURG (2020), Designing a GWAP for collecting naturally produced dialogues for low resourced languages, in Workshop on Games and Natural Language Processing, pp. 44-48.
• 30. Zulipiye YUSUPUJIANG and Jonathan GINZBURG (2021), Data collection design for dialogue systems for low-resource languages, Conversational Dialogue Systems for the Next Decade, pp. 387-392.
• 31. Zulipiye YUSUPUJIANG and Jonathan GINZBURG (2022), UgChDial: A Uyghur chat-based dialogue corpus for response space classification, in Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 3140-3149.

Identyfikatory

DOI

10.15398/jlm.v12i1.372