Modern natural language processing
CS-552 / 8 credits
Teacher: Bosselut Antoine
Language: English
Withdrawal: It is not allowed to withdraw from this subject after the registration deadline.
Summary
Natural language processing is ubiquitous in modern intelligent technologies, serving as a foundation for language translators, virtual assistants, search engines, and many more. In this course, students will learn algorithmic tools for tackling problems in modern NLP.
Content
This course includes lectures, assignments, a paper review and a project. In lectures, we will cover the foundations of modern methods for natural language processing, such as word embeddings, recurrent neural networks, transformers, pretraining, and how they can be applied to important tasks in the field, such as machine translation and text classification. We will also cover issues with these state-of-the-art approaches (such as robustness, interpretability, sensitivity), identify their failure modes in different NLP applications, and discuss analysis and mitigation techniques for these issues.
In assignments, students will be evaluated on their ability to implement methods learned in class on closed-form problems developed by the course staff. In their project, students will be expected to apply techniques learned in lecture to an open problem of their choosing. They will formulate the problem as an NLP task, propose a suitable evaluation to measure their progress, develop a model to solve the task, and provide analysis of the strengths and weaknesses of their method.
This course is of interest to MS / PhD student interested in modern methods and issues in natural language processing, both from a research and applied perspective. Senior undergraduate students will be eligible upon petition to the course instructor.
Learning Prerequisites
Recommended courses
- CS-233a or CS-233b Introduction to machine learning
- CS-433 Machine learning
Important concepts to start the course
- Python programming
- Probability and Statistics
- Linear Algebra
- Machine Learning concepts
Learning Outcomes
By the end of the course, the student must be able to:
- Define basic problems and tasks in natural language processing (e.g., machine translation, summarization, text classification, language generation, sequence labeling, informaiton extraction, question answering)
- Implement common modern approaches for tackling NLP problems and tasks (embeddings, recurrent neural models, attentive neural models) and how to train them
- Understand failure modes of these models and learning algorithms (e.g., robustness, interpretability/explainability, bias, evaluation)
- Review academic research papers and understand their contributions, strengths, and weaknesses according to the principles learned in lecture
- Complete a project that applies these algorithms to a real-world NLP problem, where they will define a task, evaluation, model implementation, and analyze the shortcomings of their approach
Teaching methods
- Lectures
- Lab sessions
- Paper reading
- Course project
Expected student activities
- Attend lectures and participate in class
- Complete homework assignments
- Complete a review of a research paper of their choosing published at an NLP conference over the last 5 years
- Complete a project of their choosing (agreed upon with course supervisor) : complete a project proposal outlining topic and evaluation plan; submit two project milestones; submit final project report; present project findings to committee of course instructor and TAs.
Assessment methods
- Assignments (40%)
- Group Project (60%)
Supervision
| Office hours | Yes |
| Assistants | Yes |
| Forum | Yes |
In the programs
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
- Semester: Spring
- Exam form: During the semester (summer session)
- Subject examined: Modern natural language processing
- Lecture: 3 Hour(s) per week x 14 weeks
- Exercises: 2 Hour(s) per week x 14 weeks
- Project: 1 Hour(s) per week x 14 weeks
Reference week
| Mo | Tu | We | Th | Fr | |
| 8-9 | |||||
| 9-10 | |||||
| 10-11 | |||||
| 11-12 | STCC - Cloud C | ||||
| 12-13 | |||||
| 13-14 | STCC - Cloud C | CE6 | |||
| 14-15 | CE1 CE1104 | ||||
| 15-16 | |||||
| 16-17 | |||||
| 17-18 | |||||
| 18-19 | |||||
| 19-20 | |||||
| 20-21 | |||||
| 21-22 |
Légendes:
Lecture
Exercise, TP
Project, other
Wednesday, 11h - 13h: Lecture STCC - Cloud C
Wednesday, 13h - 14h: Project, other STCC - Cloud C
Thursday, 13h - 14h: Lecture CE6