Large language models in question-and-answer systems: from transformer to your own chatbot

Large language models (LLMs) have become an important area in the field of artificial intelligence and computational linguistics. They are used for text generation, translation, answering questions, and many other natural language processing tasks. Modern LLMs, such as GPT‑3, LLaMa, and Mistral, can generate coherent and high‑quality texts. One typical use case for LLMs is question‑answering systems, where the language model acts as an expert providing answers to user questions. However, the responses generated by the model sometimes contain so‑called hallucinations—instances where the model generates made‑up facts, events, or concepts. In addition, there is a related problem: the model may simply lack the required knowledge. One way to address these issues is to use information retrieval systems: relevant text fragments are extracted from them and provided as prompts/support to the language model. For these reasons, the course will cover modern approaches to information retrieval and question answering based on LLMs, as well as the current technology for combining LLMs with information retrieval—Retrieval Augmented Generation (RAG).

The main goal of the course is to provide students with knowledge of both the theoretical foundations, architecture, and methods for training and applying large language models. The course has a substantial practical component and aims to enable students to:

1. Understand the principles of how transformers work;

2. Master methods for applying, training, and fine‑tuning LLMs for various natural language processing tasks;

3. Study ways to combine information retrieval and LLMs in the task of generating answers to questions, in particular the Retrieval Augmented Generation (RAG) technology;

4. Develop their own projects using LLMs and evaluate their effectiveness.

The course program includes 12 sessions: 11 lectures, 1 seminar, and 5 practical homework assignments

Requirements for students:

• Basic knowledge of machine learning;
• Basic knowledge of deep learning and PyTorch;
• Basic knowledge of natural language processing;
• Python.

Занятия проводятся в ауд. 607 (2ой ГУМ) на факультете вычислительной математики и кибернетики МГУ им. М. В. Ломоносова

В программе курса 12 занятий: 11 лекций, 1 семинар, а также 5 практических домашних заданий

Формат проведения: офлайн

Старт курса: 15 февраля 2024 года

Занятия будут проходить по четвергам с 18:00 до 19:35

Набор на курс 2023/2024 года закрыт