Wyniki wyszukiwania - BazTech

Ograniczanie wyników

1 International Journal of Electronics and Telecommunications

1 2024

Znaleziono wyników: 1

Liczba wyników na stronie

Wyniki wyszukiwania

Wyszukiwano:
w słowach kluczowych: ONNX

Sortuj według:

Ogranicz wyniki do:

Implementation of language models within an infrastructure designed for Natural Language Processing

Walkowiak Bartosz, Walkowiak Tomasz

International Journal of Electronics and Telecommunications

2024

Vol. 70, No. 1

153--159

This paper explores cost-effective alternatives for resource-constrained environments in the context of language models by investigating methods such as quantization and CPUbased model implementations. The study addresses the computational efficiency of language models during inference and the development of infrastructure for text document processing. The paper discusses related technologies, the CLARIN-PL infrastructure architecture, and implementations of small and large language models. The emphasis is on model formats, data precision, and runtime environments (GPU and CPU). It identifies optimal solutions through extensive experimentation. In addition, the paper advocates for a more comprehensive performance evaluation approach. Instead of reporting only average token throughput, it suggests considering the curve’s shape, which can vary from constant to monotonically increasing or decreasing functions. Evaluating token throughput at various curve points, especially for different output token counts, provides a more informative perspective.