Exploring the usability and user experience of social media apps through a text mining approach

• Autorzy: Baj-Rogowska Anna , Sikorski Marcin

Identyfikatory

DOI

10.2478/emj-2023-0007

• Języki publikacji: EN

Abstrakty

EN

This study aims to evaluate the applicability of a text mining approach for extracting UUX-related issues from a dataset of user comments and not to evaluate the Instagram (IG) app. This study analyses textual data mined from reviews in English written by IG mobile application users. The article’s authors used text mining (based on the LDA algorithm) to identify the main UUX-related topics. Next, they mapped the identified topics with known theoretical constructs to place them in their nomological network relevant to the usability (the 5Es framework by Quesenbery) and UX (the Honeycomb model by Morville). Finally, to expand the study with an emotional diagnosis, sentiment analysis was performed on two levels: (i) for each recognised topic, and (ii) for the full dataset to uncover general insights into users’ emotions within all reviews. The case study of the IG app confirms the usefulness of user feedback data for software development and points out that the review data have the potential for the early detection of frustration and negative feelings introduced during the use of the application. Conducting conventional UUX evaluations with users is problematic since they are remotely located, and the user-generated content of a social app undergoes continuous and frequent changes. Thus, the consecutive stages of the proposed methodology, based on text mining algorithms, constitute a proposed framework for examining the user-perceived quality projection of applications from user feedback, and they are the main contribution of this article. The used approach can be valuable for helping developers, designers and researchers to reveal user problems and fulfil user satisfaction regarding UUX aspects for specific software features.

Słowa kluczowe

EN

usability; user experience; 5Es model; honeycomb model; text mining; mobile apps

PL

użyteczność; doświadczenie użytkownika; model 5E; model plastra miodu; eksploracja tekstu; aplikacja mobilna

• Wydawca: Oficyna Wydawnicza Politechniki Białostockiej
• Czasopismo: Engineering Management in Production and Services
• Rocznik: 2023
• Tom: Vol. 15, no. 1
• Strony: 86--105
• Opis fizyczny: Bibliogr 76 poz., tab., wykr.

Twórcy

autor

Baj-Rogowska Anna

• Gdańsk University of Technology, Poland

• https://orcid.org/0000-0002-8694-4461

autor

Sikorski Marcin

• Gdańsk University of Technology, Poland

• https://orcid.org/0000-0001-7930-7790

Bibliografia

• Abbasi, A., Sarker, S., & Chiang, R. H. (2016). Big data research in information systems: Toward an inclusive research agenda. Journal of the Association for Information Systems, 17(3).
• Adikari, S., McDonald, C., & Campbell, J. (2011). A design science framework for designing and assessing user experience. In International Conference on Human-Computer Interaction (pp. 25–34). Berlin, Heidelberg: Springer.
• Baj-Rogowska, A. (2020). Evaluation of a company’s image on social media using the Net Sentiment Rate. In E. Lechman & M. Popowska (Eds.), Society and Technology Opportunities and Challenges (pp. 202–218). doi: 10.4324/9780429278945
• Bakiu, E., & Guzman, E. (2017). Which feature is unusable? Detecting usability and user experience issues from user reviews. In 2017 IEEE 25th international requirements engineering conference workshops (REW) (pp. 182-–187). IEEE.
• Battarbee, K. (2003). Co-experience: the social user experience. In CHI ‘03 Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘03) (pp. 730– 731). New York, NY, USA: Association for Computing Machinery. doi: 10.1145/765891.765956
• Berenbach, B., Paulish, D. J., Kazmeier, J., & Rudorfer, A. (2009). Software & Systems Requirements Engineering: In Practice. Retrieved from https://www.accessengineeringlibrary.com/content/book/9780071605472
• Bevan, N. (2008). Classifying and selecting UX and usability measures. In Proceedings of Meaningful Measures: Valid Useful User Experience Measurement (VUUM), 5th COST 294-MAUSE Open Workshop (pp. 13–18). Reykjavik, Iceland.
• Bilgihan, A. (2016). Gen Y customer loyalty in online shop¬ping: An integrated model of trust, user experience and branding. Computers in Human Behavior, 61, 103-113.
• Blythe, M., Overbeeke, K., Monk, A., & Wright, P. (2003). Funology: from usability to enjoyment, Kluwer Academic Publishers.
• Boyd-Graber, J., Mimno, D., & Newman, D. (2014). Care and feeding of topic models: Problems, diagnostics, and improvements. In E. M. Airoldi, D. Blei, E. A. Erosheva, & S. E. Fienberg (Eds.), Handbook of mixed membership models and their applications (pp. 3–34). Boca Raton: CRC Press.
• Cardonen, B., Di Martino, F., & Senatore, S. (2021).Improving the emotion-based classification by exploiting the fuzzy entropy in FCM clustering. International Journal of Intelligent Systems, 36(11), 6944-6967. doi: 10.1002/int.22575
• Carreño, L. V., & Winbladh, K. (2013). Analysis of User Comments: An Approach for Software Requirements Evolution. In Proceedings of the 2013 International Conference on Software Engineering ser. ICSE’13 (pp. 582–591).
• Chang, H. H., & Chen, S. W. (2009). Consumer perception of interface quality, security, and loyalty in electronic commerce. Information & Management, 46(7), pp. 411-417.
• Chen, J. V., Yen, D. C., Pornpriphet, W., & Widjaja, A. E. (2015). E-commerce website loyalty: A cross cultural comparison. Information Systems Frontiers, 17(6), 1283-1299.
• Cockburn A., Quinn P., & Gutwin C. (2015). Examining the Peak-End Effects of Subjective Experience. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15) 9pp. 357–366). New York, NY, USA: Association for Computing Machinery. doi: 10.1145/2702123.2702139
• Cuadrado, F., & Dueñas, J. C. (2012). Mobile application stores: success factors, existing approaches, and future developments. IEEE Communications Magazine, 50(11), 160-167.
• Debortoli, S., Müller, O., Junglas, I., & vom Brocke, J. (2016). Text Mining For Information Systems Re¬searchers: An Annotated Topic Modeling Tutorial. Communications of the Association for Information Systems, 39. doi: 10.17705/1CAIS.03907
• Digitalinformationworld (2022). Retrieved from https://www.digitalinformationworld.com/2022/08/instagram-rumored-to-be-following-in.html
• Eid, M. I. (2011). Determinants of e-commerce customer satisfaction, trust, and loyalty in Saudi Arabia. Journal of Electronic Commerce Research, 12(1), 78.
• Fashionunited (2022). Retrieved from https://fashionunited.uk/news/fashion/instagram-after-a-backlash-pauses-changes-to-its-algorithm/2022080864536
• Følstad, A., & Rolfsen, R. K. (2006). Measuring the Effect of User Experience Design Changes in ECommerce Web Sites: A Case on Customer Guidance. In Proceedings of the 2nd COST294-MAUSE International Open Workshop (pp. 10–15). Oslo, Norway.
• Forlizzi, J., & Battarbee, K. (2004). Understanding experience in interactive systems. In Proceedings of the 2004 Conference on Designing Interactive Systems (DIS 04) Processes, Practices, Methods, and Techniques. New York: ACM.
• Fu, B., Lin, J., Li, L., Faloutsos, C., Hong, J., & Sadeh, N. (2013). Why people hate your app: making sense of user feedback in a mobile app store. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’13 (p. 1276). Presented At the 19th ACM SIGKDD International Conference. Chicago, Illinois, USA: ACM Press.
• Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences, 101(suppl 1), 5228-5235. doi: 10.1073/ pnas.0307752101
• Guzman, E., Aly, O., & Bruegge, B. (2015). Retrieving Diverse Opinions from App Reviews. In Proceedings of the 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement ser. ESEM’15 (pp. 1–10).
• Häkkilä, J., Korpipää, P., Ronkainen, S., & Tuomela, U. (2005). Interaction and end-user programming with a context-aware mobile application. In Proc. IFIP Conf. Hum.-Comput. Interact (pp. 927–937). Berlin, Germany: Springer.
• Hartson, R., & Pyla, P. S. (2012). The UX Book. Process and Guidelines for Ensuring a Quality User Experience. Elsevier.
• Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products, Human-Computer Interaction, 19(4), 319-349.
• Hassenzahl, M. (2008). User Experience (UX): Towards an Experiential Perspective on Product Quality. In Proceedings of the 20th Conference on l’Interaction Homme-Machine (pp. 11–15). New York, NY, USA: Association for Computing Machinery.
• Hassenzahl, M., & Tractinksy, N. (2006). User experience: a research agenda. Behaviour and Information Technology, 25(2), 91-97.
• Hedegaard, S., & Simonsen, J. G. (2013). Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2089–2098).
• Hu, N., Pavlou P. A., & Zhang J. (2006). Can online reviews reveal a product’s true quality?: Empirical findings and analytical modeling of online word-of-mouth communication. In Proceedings of the 7th ACM Conference on Electronic Commerce (EC) (pp. 324–330).
• IEEE Std 610.12-1990 (1990). IEEE standard glossary of software engineering terminology. Institute of Electri¬cal and Electronics Engineers, Inc.
• ISO 9241-11:2018 (2018). Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. International Standards Organization.
• ISO 9241-210: 2019 (2010). Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. International Standards Organization.
• ISO/IEC 25010:2011 (2011). Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. International Standards Organization.
• ISO/IEC 9126-1:2001 (2001). Software engineering — Product quality — Part 1: Quality model. International Standards Organization.
• ISO/IEC, 9241-11:1998 (1998). Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11 Guidance on usability. 1998: ISO/IEC 9241- 11: 1998 (E).
• Jacob, C., & Harrison, R. (2013). Retrieving and analyzing mobile apps feature requests from online reviews. In Proceedings of the Working Conference on Mining Software Repositories ser. MSR’13 (pp. 41–44).
• Jiménes, K., Pincay, J., Villavicencio, M., & Jiménez, A. (2018). Looking for usability and functionality issues: a case study. In International Conference on Information Technology & Systems (pp. 948–958). Cham: Springer.
• Korhan, O., & Ersoy, M. (2016). Usability and functionality factors of the social network site application users from the perspective of uses and gratification theory. Qual Quant, 50, 1799-1816. doi: 10.1007/s11135-015-0236-7
• Kumar, B. A., & Mohite, P. (2018). Usability of mobile learning applications: A systematic literature review. Journal of Computers in Education, 5(1), 1-17.
• Liu, R., Cao, J., Zhang, K., Gao, W., Liang, J., & Yang, L. (2018). When privacy meets usability: Unobtrusive privacy permission recommendation system for mobile apps based on crowdsourcing. IEEE Transactions on Services Computing, 11(5), pp. 864–878.
• Maalej, W., & Pagano, D. (2011). On the socialness of software. In 2011 IEEE Ninth International Conference on Dependable Autonomic and Secure Computing (DASC) (pp. 864–871).
• McCarthy, J., & Wright, P. (2004). Technology as Experience. Cambridge, MA, USA: MIT Press.
• McNamara, N., & Kirakowski, J. (2006). Functionality, Usability, and User Experience: Three Areas of Concern. Interactions, 13, 26-28.
• Meuthia, R. F., Ferdawati, F., & Gustati, G. (2021). Analisis User Experience Pada Aplikasi E-Kelurahan Menggunakan Model UX Honeycomb. Journal Akuntansi Keuangan dan Bisnis, 14(2), 342-351.
• Morville, P. (2004). User Experience Design. Semantic Studios: Interwingled. Retrieved from http://semantic-studios.com/publications/semantics/000029.php
• Morville, P. (2016). User experience honeycomb. Semantic Studios: Interwingled. Retrieved from https://inter-twingled.org/user-experience-honeycomb/
• Nakamura, W. T., de Oliveira, E. C., de Oliveira, E. H. T., Redmiles, D., & Conte, T. (2022). What factors affect the UX in mobile apps? A systematic mapping study on the analysis of app store reviews, Journal of Systems and Software, 193. doi: 10.1016/j.jss.2022.111462
• Nielsen Norman Group. (2021). Social Media UX: User-Centered Social Strategies. Retrieved from https://www.nngroup.com/reports/social-media-user-expe¬rience/
• Norman, D. A. (1998). The invisible computer. Cambridge, Mass: MIT Press.
• Omotosho, B. S. (2021). Analysing user experience of mobile banking applications in Nigeria: A text mining approach. CBN Journal of Applied Statistics, 12(1), 77-108.
• Pagano, D., & Bruegge, B. (2013). User Involvement in Software Evolution Practice: A Case Study. In Proceedings of the 35th International Conference on Software Engineering. San Francisco, CA: IEEE.
• Pagano, D., & Maalej, W. (2013). User Feedback in The Appstore: an Empirical Study. In Proceedings 2013 21st IEEE International Requirements Engineering Conference ser. RE’13 (pp. 125–134).
• Park, J., Han, S. H., Kim, H. K., Cho, Y., & Park, W. (2013). Developing elements of user experience for mobile phones and services: survey, interview, and observation approaches. Human Factors and Ergonomics in Manufacturing & Service Industries, 23(4), 279-293.
• Pereira, R., Baranauskas M. C. C., & da Silva, S. R. P. (2010). A Discussion on Social Software: Concept, Building Blocks and Challenges. International Journal for Infonomics (IJI), 3(4).
• Porat, T., & Tractinsky, N. (2012). It’s a pleasure buying here: The effects of web-store design on consumers’ emotions and attitudes. Human–Computer Interaction, 27(3), 235-276.
• Portugal, R. L., & Sampaio do Prado Leite, J. C. (2018). Usability Related Qualities Through Sentiment Analysis. In 1st International Workshop on Affective Computing for Requirements Engineering (AffectRE) (pp. 20–26).
• Quesenbery, W. (2004). Balancing the 5Es: Usability. Cutter IT Journal, 17(2), pp. 4-11.
• Quesenbery, W. (2014). The five dimensions of usability. In Content and complexity 9pp. 93–114). Routledge.
• Rajkumar, A. V., Suresh, C. E., Madhavan, V., & Narasimha Murthy, M. N. (2010). On finding the natural number of topics with Latent Dirichlet Allocation: Some observations. In M. J. Zaki, J. X. Yu, B. Ravindran & V. Pudi (Eds.). Advances in knowledge discovery and data mining (pp. 391–402). Berlin Heidelberg: Springer. doi: 10.1007/978-3-642-13657-3_43
• Saavedra, M. J., Rusu, C., Quiñones, D., & Roncagliolo, S. (2019). A Set of Usability and User eXperience Heuristics for Social Networks. In G. Meiselwitz (Eds.), Social Computing and Social Media. Design, Human Behavior and Analytics. HCII 2019. Lecture Notes in Computer Science, 11578. Springer, Cham. doi: 10.1007/978-3-030-21902-4_10.
• Sánchez, J. L. G., Vela, F. L. G., Simarro, F. M., & Padilla-Zea, N. (2012). Playability: analysing user experience in video games. Behaviour & Information Technology, 31(10), 1033-1054.
• Sauer, J., Sonderegger, A., & Schmutz, S. (2020). Usability, User Experience and Accessibility: Towards an Integrative Model. Ergonomics, 63, 1207-1220.
• Sharp, H., Rogers, Y., & Preece, J. (2023). Interaction design: beyond human-computer interaction. 6th Edition. London: John Wiley.
• Shneiderman, B., & Plaisant, C. (2016). Designing the user interface: strategies for effective human-computer interaction. Boston: Pearson Education.
• Sikorski, M. (2008). HCI and the Economics of User Experience. In E. Law, E. Hvannberg & G. Cockton (Eds.), Maturing Usability (pp. 318–343. London: Springer-Verlag.
• Smith, G. (2007). Social Software Building Blocks. Retrieved from http://nform.ca/publications/social-software-building-block
• Souza, P. C. D., & Maciel, C. (2015). Legal issues and user experience in ubiquitous systems from a privacy perspective. In International Conference on Human Aspects of Information Security, Privacy, and Trust (pp. 449–460). Cham: Springer.
• Sutcliffe, A. (2009). Designing for User Engagement: Aesthetic and Attractive User Interfaces. San Rafael, CA, USA: Morgan & Claypool Publishers.
• Väänänen-Vainio-Mattila, K., Wäljas, M., Ojala, J., & Segerståhl, K. (2010). Identifying drivers and hindrances of social user experience in web services. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2499–2502).
• Vivakaran, M. V., & Neelamalar, M. (2015). From Spectators to Users: A study on the transformation caused by the Social media in the Indian Media Context.
• Wang, Y., & Brennan, R. (2017). Building high usability consumer-oriented linked data mobile apps. In Proceedings Irish Conference on Artificial Intelligence and Cognitive Science (AICS) (pp. 1–12). Dublin, Republic of Ireland.
• Weichbroth, P., & Baj-Rogowska, A. (2019). Do online reviews reveal mobile application usability and user experience? The case of WhatsApp. In Federated Conference on Computer Science and Information Systems (pp. 747–754).
• Žižka, J., Dařena, F., & Svoboda, A. (2020). Text Mining with Machine Learning: Principles and Techniques. Taylor & Francis Group, CRC Press.