Wyniki wyszukiwania - BazTech

Ograniczanie wyników

1 Annals of Computer Science and Information Systems

1 2024

Znaleziono wyników: 1

Liczba wyników na stronie

Wyniki wyszukiwania

Sortuj według:

Ogranicz wyniki do:

Impact of Spelling and Editing Correctness on Detection of LLM-Generated Emails

Gryka Paweł, Gradoń Kacper, Kozłowski Marek, Kutyła Miłosz, Janicki Artur

Annals of Computer Science and Information Systems

2024

Vol. 39

603--608

In this paper, we investigated the impact of spelling and editing correctness on the accuracy of detection if an email was written by a human or if it was generated by a language model. As a dataset, we used a combination of publicly available email datasets with our in-house data, with over 10k emails in total. Then, we generated their “copies'' using large language models (LLMs) with specific prompts. As a classifier, we used random forest, which yielded the best results in previous experiments. For English emails, we found a slight decrease in evaluation metrics if error-related features were excluded. However, for the Polish emails, the differences were more significant, indicating a decline in prediction quality by around 2% relative. The results suggest that the proposed detection method can be equally effective for English even if spelling- and grammar-checking tools are used. As for Polish, to compensate for error-related features, additional measures have to be undertaken.