Human language technology: applications to information access

The methods, presented in this course, enable users to access to textual information across three types of barriers: the quantity barrier (large repositories), the cross-lingual barrier (different languages), and the subjective barrier (opinions and interactions).

After a brief introduction to the stages of natural language processing and text tokenization, the course will present through lectures and practical work (50% of the time each) the following approaches that overcome the three barriers to information access:

• The quantity barrier: vector space models for information retrieval; word vectors and non-contextual embeddings using RNNs and Transformers; text embeddings and vector databases; basics of question answering and retrieval augmented generation (RAG).
• The cross-lingual barrier: machine translation (MT) with n-gram models; decoding; recurrent neural models with attention; the Transformer for MT; large language models and their use for translation; evaluation of MT and translation biases.
• The subjective barrier: neural models for sentiment analysis; text generation using Transformers-based decoders; fine-tuning of LLMs (data and algorithms); instructed LLMs as charbots or problem solvers; prompt engineering and agentic AI.
• Power consumption, and ethical issues related to LLMs.