Improving utilization of lexical knowledge in natural language inference

• Autorzy: Chłędowski, Jakub , Wesołowski, Tomasz , Jastrzębski, Stanisław
• Języki publikacji: EN

Abstrakty

EN

Natural language inference (NLI) is a central problem in natural language processing (NLP) of predicting the logical relationship between a pair of sentences. Lexical knowledge, which represents relations between words, is often important for solving NLI problems. This knowledge can be accessed by using an external knowledge base (KB), but this is limited to when such a resource is accessible. Instead of using a KB, we propose a simple architectural change for attention based models. We show that by adding a skip connection from the input to the attention layer we can utilize better the lexical knowledge already present in the pretrained word embeddings. Finally, we demonstrate that our strategy allows to use an external source of knowledge in a straightforward manner by incorporating a second word embedding space in the model.

Słowa kluczowe

EN

natural language processing; natural language inference; representation learning; word embeddings; machine learning; deep learning

• Czasopismo: Schedae Informaticae
• Rocznik: 2018
• Tom: Vol. 27

Twórcy

autor

Chłędowski, Jakub

• Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland

autor

Wesołowski, Tomasz

• Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland

autor

Jastrzębski, Stanisław

• Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).