Systems for data management and data science

Summary

This is a course for students who want to understand modern large-scale data analysis systems and database systems. The course covers fundamental principles for understanding and building systems for managing and analyzing large amounts of data. It covers a wide range of topics and technologies.

Content

Learning Prerequisites

Required courses

• CS-107 Introduction to programming
• CS-214 Software construction
• CS-300 Data-Intensive Systems
• CS-202 Computer systems

or equivalent courses

Important concepts to start the course

• Knowledge of algorithms and data structures.
• Scala and/or Java programming languages will be used throughout the course. Programming experience in one of these languages is strongly recommanded.
• Basic knowledge or computer networking and distributed systems.

Learning Outcomes

By the end of the course, the student must be able to:

• Understand how to design big data analytics systems using state-of-the-art infrastructures for horizontal scaling, e.g., Spark
• Implement algorithms and data structures for streaming data analytics
• Decide between different storage models based on the offered optimizations enabled by each modl and the expected query workload
• Compare concurrency control algorithms, and algorithms for distributed data management
• Configure systems parameters, data layouts, and application designs for database systems
• Develop data-parallel analytics programs that make us of modern clusters and cloud offerings to scale up to very large workloads
• Analyze the trade-offs between various approaches to large-scale data management and analytics, depending on efficiency, scalability, and latency needs
• Understand in detail the design big data analytics systems using state-of-the-art infrastructures for horizontal scaling, e.g., Spark
• Understand the advantage and disavantages of different storage models for a given workload, based on the offered optimization enabled by each model and the workload characteristics

Teaching methods

Lectures, project, homework, exercises and practical work

Expected student activities

• Complete the course projects/labs, which provide practical experience with the concepts covered in the course and prepare students for the corresponding practical questions in the final exam.
• Study all material provided and recommended during the course.
• Take the final written exam.

Assessment methods

The final grade is based entirely (100%) on the final written exam.

The final exam will assess both the theoretical and practical aspects of the course. Questions similar in nature to the work carried out in the course projects/labs may account for up to the weighting previously allocated to the projects/labs in past editions of the course. The projects/labs themselves are not graded and do not contribute directly to the final grade.

Supervision

Resources

Bibliography

J. Hellerstein & M. Stonebraker, Readings in Database Systems, 4th Edition, 2005
R. Ramakrishnan & J. Gehrke: "Database Management Systems", McGraw-Hill, 3rd Edition,
2002.
A. Rajaraman & J. Ullman: "Mining of Massive Datasets", Cambridge Univ. Press, 2011.

Ressources en bibliothèque

• Find the references at the Library

Moodle Link

• https://go.epfl.ch/CS-460