BERT-CLSTM model for the classification of Moroccan commercial courts verdicts

• Autorzy: El Moussaoui Taoufiq , Chakir Loqman

Identyfikatory

DOI

10.15439/2023F3561

• Języki publikacji: EN

Abstrakty

EN

The exponential growth of data generated by the Moroccan commercial court system, coupled with the manual archiving of legal documents, has led to increasingly complex information access. As data classification becomes imperative, researchers are exploring automatic language processing techniques and refining text classification methods. In this study, we propose a BERT-CLSTM model for the classification of Moroccan commercial court verdicts. By adding a Convolutional Long Short-Term Memory Network to the task-specific layers of BERT, our model can get information on important fragments in the text. In addition, we input the representation along with the output of the BERT into the transformer encoder to take advantage of the self-attention mechanism and finally get the representation of the whole text through the transformer. The proposed model outperformed the compared baselines and achieved good results by getting an F-measure value of 93.61%.

Słowa kluczowe

EN

computer science; law; computational modeling; text categorization; refining; manual; writing

PL

informatyka; prawo; modelowanie obliczeniowe; kategoryzacja tekstu; podręcznik; pisanie

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2023
• Tom: Vol. 35
• Strony: 281--284
• Opis fizyczny: Bibliogr. 14 poz., tab., wz., wykr.

Twórcy

autor

El Moussaoui Taoufiq

• LISAC Laboratory, Faculty of Sciences Dhar El Mahraz Sidi Mohamed Ben Abdellah University Fez, Morocco

• https://orcid.org/0000-0003-4879-7111

autor

Chakir Loqman

• LISAC Laboratory, Faculty of Sciences Dhar El Mahraz Sidi Mohamed Ben Abdellah University Fez, Morocco

• https://orcid.org/0000-0002-8261-9370

Bibliografia

• 1. C. Blake, “Text Mining,” Annual review of information science and technology, vol. 45, 2011, pp. 121–155.
• 2. X. Zhang, J. Zhao, Y. LeCun, “Character-level convolutional networks for text classification,” Advances in Neural Information Processing Systems, 2015.
• 3. A. Conneau, H. Schwenk, L. Barrault, Y. LeCun, “Very deep convolutional networks for text classification,” Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 1, 2017, pp. 1107–1116.
• 4. Y. Kim, “Convolutional neural networks for sentence classification,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), vol. 1, 2014, pp. 1746–1751.
• 5. K. Sheng Tai, R. Socher, C.D. Manning, “Improved semantic representations from tree-structured long short-term memory networks,” Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, vol. 1, 2015, pp. 1556–1566.
• 6. M. Huang, Q. Qian, X. Zhu, “Encoding syntactic knowledge in neural networks for sentiment classification,” ACM Transactions on Information Systems, vol. 35, 2017, pp. 1–27.
• 7. H. El Rifai, L. Al Qadi, A. Elnagar, “Arabic text classification: the need for multi-labeling systems,” Neural Computing and Applications, vol. 34, 2022, pp. 1135–1159.
• 8. M. AL-Qurishi, S. AlQaseemi, R. Soussi, “AraLegal-BERT: A pretrained language model for Arabic Legal text,” Proceedings of the Natural Legal Language Processing Workshop 2022, 2022, pp. 338–344.
• 9. F. El-Alami, A. El Mahdaouy, S.O. El Alaoui, N. En-Nahnahi, “A deep autoencoder-based representation for Arabic text categorization,” Journal of Information and Communication Technology, vol. 3, 2020, pp. 381–398.
• 10. W.H.G. Gwad, I.M.I. Ismael, Y. Gultepe, “Twitter Sentiment Analysis Classification in the Arabic Language using Long Short-Term Memory Neural Networks,” International Journal of Engineering and Advanced Technology, vol. 9, 2020, pp. 235–239.
• 11. M. Alhawarat, A.O. Aseeri, “A Superior Arabic Text Categorization Deep Model (SATCDM),” IEEE Access, vol. 8, 2020, pp. 24653–24661.
• 12. M. Galal, M.M. Madbouly, A. El-Zoghby, “Classifying Arabic text using deep learning,” Journal of Theoretical and Applied Information Technology, vol. 97, 2019, pp. 3412–3422.
• 13. S. Boukil, M. Biniz, F. El Adnani, L. Cherrat, A.E. El Moutaouakkil, “Arabic Text Classification Using Deep Learning Technics,” International journal of grid and distributed computing, vol. 11, 2018, pp. 103–114.
• 14. R. Al-khurayji and A. Sameh, “An Effective Arabic Text Classification Approach Based on Kernel Naive Bayes Classifier,” International Journal of Artificial Intelligence and Applications, vol. 8, 2017, pp. 01–10.

Uwagi

1. This work was supported by the Ministry of Higher Education, Scientific Research and Innovation, the Digital Development Agency (DDA) and the CNRST of Morocco [Alkhawarizmi/2020/36].

2. Main Track Short Papers

3. Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).