Sentiment-aware enhancements of pagerank-based citation metric, impact factor, and h-index for ranking the authors of scholarly articles

• Autorzy: Gupta Shikha , Kumar Animesh

Identyfikatory

DOI

10.7494/csci.2024.25.2.6042

• Języki publikacji: EN

Abstrakty

EN

Heretofore, the only way to evaluate an author has been frequency-based citation metrics that assume citations to be of a neutral sentiment. However, considering the sentiment behind citations aids in a better understanding of the viewpoints of fellow researchers for the scholarly output of an author. We present sentiment-enhanced alternatives to three conventional metrics namely Impact Factor, H-index, and Page Rank-based index. The proposal studies the impact of the proposed metrics on the ranking of authors. We experimented with two datasets, collectively comprising almost 20,000 citation sentences. The evaluation of the proposed metrics revealed a significant impact of sentiments on author ranking, evidenced by a weak Kendall coefficient for the Author Impact Factor and H-index. However, the Page Rank-based metric showed a moderate to strong correlation, due to its prestige-based attributes. Further more, a remarkable Rank-biased deviation exceeding 28% was seen in all cases, indicating a stronger rank deviation in top-ordered ranks.

Słowa kluczowe

EN

Kendall coefficient; rank biased distance; bibliometrics; scientometrics

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2024
• Tom: T. 25 (2)
• Strony: 173--209
• Opis fizyczny: Bibliogr. 39 poz., rys., tab.

Twórcy

autor

Gupta Shikha

• Shaheed Sukhdev College of Business Studies, New Delhi, India

autor

Kumar Animesh

• Shaheed Sukhdev College of Business Studies, New Delhi, India

Bibliografia

• [1] Abu-Jbara A., Ezra J., Radev D.: Purpose and polarity of citation: Towards NLP-based bibliometrics. In: Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: Human language technologies, pp. 596–606, 2013.
• [2] Akoglu H.: User’s guide to correlation coefficients, Turkish Journal of Emergency Medicine, vol. 18(3), pp. 91–93, 2018. doi: 10.1016/j.tjem.2018.08.001
• [3] Aljuaid H., Iftikhar R., Ahmad S., Asif M., Tanvir Afzal M.: Important citation identification using sentiment analysis of in-text citations, Telematics and Informatics, vol. 56, 101492, 2021. doi: 10.1016/j.tele.2020.101492.
• [4] Amjad Z., Ihsan I.: VerbNet based Citation Sentiment Class Assignment using Machine Learning, International Journal of Advanced Computer Science & Applications, vol. 11(9), 2020. doi: 10.14569/ijacsa.2020.0110973.
• [5] Athar A.: Sentiment analysis of citations using sentence structure-based features. In: S. Petrovic, E. Selfridge, E. Pitler, M. Osborne, T. Solorio (eds.), Proceedings of the ACL 2011 Student Session, pp. 81–87, Association for Computational Linguistics, Portland, OR, USA, 2011. https://aclanthology.org/P11-3015/
• [6] Baccianella S., Esuli A., Sebastiani F.: SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, D. Tapias (eds.), Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), pp. 2200–2204, European Language Resources Association (ELRA), Valletta, Malta, 2010. http://lrec-conf.org/proceedings/lrec2010/pdf/769 Paper.pdf.
• [7] Bressan M., Peserico E.: Choose the damping, choose the ranking?, Journal of Discrete Algorithms, vol. 8(2), pp. 199–213, 2010. doi: 10.1016/j.jda.2009.11.001.
• [8] Brin S., Page L.: The anatomy of a large-scale hypertextual web search engine, Computer networks and ISDN systems, vol. 30(1-7), pp. 107–117, 1998. doi: 10.1016/s0169-7552(98)00110-x.
• [9] Cohan A., Ammar W., Zuylen M.V., Cady F.: Structural Scaffolds for Citation Intent Classification in Scientific Publications. In: J. Burstein, C. Doran, T. Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019. doi: 10.18653/v1/N19-1361.
• [10] Fellbaum C.: WordNet. In: R. Poli, M. Healy, A. Kameas (eds.), Theory and applications of ontology: computer applications, pp. 231–243, Springer, Dordrecht, 2010. doi: 10.1007/978-90-481-8847-5 10
• [11] Garfield E.: Journal impact factor: a brief review, CMAJ: Canadian Medical Association Journal, vol. 161(8), pp. 979–980, 1999. https://www.cmaj.ca/content/161/8/979.short.
• [12] Ghosh S., Shah C.: Identifying Citation Sentiment and its Influence while Indexing Scientific Papers. In: Proceedings of the 53rd Hawaii International Conference on System Sciences, pp. 1–10, 2020. doi: 10.24251/hicss.2020.307
• [13] Hirsch J.E.: An index to quantify an individual’s scientific research output, Proceedings of the National Academy of Sciences, vol. 102(46), pp. 16569–16572, 2005. doi: 10.1073/pnas.0507655102.
• [14] Ihsan I., Qadir M.A.: CCRO: Citation’s Context & Reasons Ontology, IEEE Access, vol. 7, pp. 30423–30436, 2019. doi: 10.1109/access.2019.2903450.
• [15] Karim M., Missen M.M.S., Umer M., Sadiq S., Mohamed A., Ashraf I.: Citation Context Analysis Using Combined Feature Embedding and Deep Convolutional Neural Network Model, Applied Sciences, vol. 12(6), 3203, 2022. doi: 10.3390/app12063203.
• [16] Kazi P., Patwardhan M., Joglekar P.: Towards a new perspective on context based citation index of research articles, Scientometrics, vol. 107, pp. 103–121, 2016. doi: 10.1007/s11192-016-1844-2.
• [17] Kilicoglu H., Peng Z., Tafreshi S., Tran T., Rosemblat G., Schneider J.: Confirm or refute?: A comparative study on citation sentiment classification in clinical research publications, Journal of Biomedical Informatics, vol. 91, 103123, 2019. doi: 10.1016/j.jbi.2019.103123.
• [18] Kinney R.M., Anastasiades C., Authur R., Beltagy I., Bragg J., Buraczynski A., Cachola I., et al.: The Semantic Scholar Open Data Platform, ArXiv, 2023. doi: 10.48550/arXiv.2301.10140.
• [19] Kochhar S.K., Ojha U.: Index for objective measurement of a research paper based on sentiment analysis, ICT Express, vol. 6(3), pp. 253–257, 2020. doi: 10.1016/j.icte.2020.02.001.
• [20] Liu H.: Sentiment analysis of citations using word2vec, arXiv preprint arXiv:170400177, 2017. doi: 10.48550/arXiv.1704.00177.
• [21] Ma Z., Nam J., Weihe K.: Improve sentiment analysis of citations with author modelling. In: A. Balahur, E. van der Goot, P. Vossen, A. Montoyo (eds.), Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 122–127, Association for Computational Linguistics, San Diego, California, 2016. doi: 10.18653/v1/w16-0420.
• [22] MacRoberts M.H., MacRoberts B.R.: The negational reference: Or the art of dissembling, Social Studies of Science, vol. 14(1), pp. 91–94, 1984. doi: 10.1177/030631284014001006.
• [23] Malkawi R., Daradkeh M., El-Hassan A., Petrov P.: A Semantic Similarity-Based Identification Method for Implicit Citation Functions and Sentiments Information, Information, vol. 13(11), 546, 2022. doi: 10.3390/info13110546.
• [24] Mantyla M.V., Graziotin D., Kuutila M.: The evolution of sentiment analysis – A review of research topics, venues, and top cited papers, Computer Science Review, vol. 27, pp. 16–32, 2018. doi: 10.1016/j.cosrev.2017.10.002.
• [25] Mercier D., Bhardwaj A., Dengel A., Ahmed S.: SentiCite: An Approach for Publication Sentiment Analysis, arXivorg, 2019. doi: 10.48550/arXiv.1910.03498.
• [26] Nazir S., Asif M., Ahmad S., Aljuaid H., Iftikhar R., Nawaz Z., Ghadi Y.Y.: Important Citation Identification by Exploding the Sentiment Analysis and SectionWise In-Text Citation Weights, IEEE Access, vol. 10, pp. 87990–88000, 2022. doi: 10.1109/access.2022.3199420.
• [27] Raza H., Faizan M., Hamza A., Mushtaq A., Akhtar N.: Scientific Text Sentiment Analysis using Machine Learning Techniques, International Journal of Advanced Computer Science & Applications, vol. 10(12), 2019. doi: 10.14569/ijacsa.2019.0101222.
• [28] Sardar A., Pramanik P.K.D.: Estimating Authors’ Research Impact Using PageRank Algorithm. In: Data Management, Analytics and Innovation: Proceedings of ICDMAI 2021, Volume 2, pp. 471–483, Springer, 2022. doi: 10.1007/978-981-16-2937-2 29.
• [29] Teufel S., Siddharthan A., Tidhar D.: Automatic classification of citation function. In: Proceedings of the 2006 conference on empirical methods in natural language processing, pp. 103–110, 2006. doi: 10.3115/1610075.1610091.
• [30] Umer M., Sadiq S., Missen M.M.S., Hameed Z., Aslam Z., Siddique M.A., Nappi M.: Scientific papers citation analysis using textual features and SMOTE resampling techniques, Pattern Recognition Letters, vol. 150, pp. 250–257, 2021. doi: 10.1016/j.patrec.2021.07.009.
• [31] Umer M., Sadiq S., Missen M.M.S., Hameed Z., Aslam Z., Siddique M.A., Nappi M.: Scientific papers citation analysis using textual features and SMOTE resampling techniques, Pattern Recognition Letters, vol. 150, pp. 250–257, 2021. doi: 10.1016/j.patrec.2021.07.009.
• [32] Valenzuela M., Ha V., Etzioni O.: Identifying Meaningful Citations. In: Proceedings of AAAI Workshop Papers 2015, Scholarly Big Data: AI Perspectives, Challenges, and Ideas, vol. 15, pp. 21–26, 2015. https://cdn.aaai.org/ocs/ws/ws0121/10185-46008-1-PB.pdf.
• [33] Visser R., Dunaiski M.: Sentiment and intent classification of in-text citations using BERT. In: Proceedings of 43rd Conference of the South African Institute of Computer Scientists and Information Technologists, vol. 85, pp. 129–145, 2022. doi: 10.29007/wk21.
• [34] Wang M., Zhang J., Jiao S., Zhang X., Zhu N., Chen G.: Important citation identification by exploiting the syntactic and contextual information of citations, Scientometrics, vol. 125, pp. 2109–2129, 2020. doi: 10.1007/s11192-020-03677-1.
• [35] Webber W., Moffat A., Zobel J.: A similarity measure for indefinite rankings, ACM Transactions on Information Systems (TOIS), vol. 28(4), pp. 1–38, 2010. doi: 10.1145/1852102.1852106.
• [36] Xu L., Ding K., Lin Y., Zhang C.: Does citation polarity help evaluate the quality of academic papers?, Scientometrics, pp. 4065–4087, 2023. doi: 10.1007/s11192-023-04734-1.
• [37] Yan E., Chen Z., Li K.: Authors’ status and the perceived quality of their work: Measuring citation sentiment change in nobel articles, Journal of the Association for Information Science and Technology, vol. 71(3), pp. 314–324, 2020. doi: 10.1002/asi.24237.
• [38] Yousif A., Niu Z., Chambua J., Khan Z.Y.: Multi-task learning model based on recurrent convolutional neural networks for citation sentiment and purpose classification, Neurocomputing (Amsterdam), vol. 335, pp. 195–205, 2019. doi: 10.1016/j.neucom.2019.01.021.
• [39] Yousif A., Niu Z., Tarus J.K., Ahmad A.: A survey on sentiment analysis of scientific citations, Artificial Intelligence Review, vol. 52, pp. 1805–1838, 2019. doi: 10.1007/s10462-017-9597-8.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).