Coursebooks 2018-2019

Performance evaluation

Lecturer(s) :

Le Boudec Jean-Yves

English

Remarque

cours biennal donné les années paires

Summary

In this course you will learn the methods and techniques that are used to perform a good performance evaluation during a research or development project.

Content

MethodologyA Performance Evaluation Methodology. The scientific method. Dijkstra and Occam's principle.
Statistics and Modeling.
Statistics and modeling, why and how. Comparing systems using sampled data. Regression models. Factorial analysis. Stochastic load and system models. Load forecasting. The Box-Jenkins method.
Practicals.
Using a statistics package (Matlab). Measurements. Discrete event simulation. Stationarity and Steady State. Analysis of simulation results. Perfect Simulations.
Elements of a Theory of Performance.Performance of systems with waiting times. Utilization versus waiting times. Operational laws. Little's formula. Forced flows.law. Stochastic modeling revisited. The importance of the viewpoint. Palm calculus. Application to Simulation Performance patterns in complex systems. Bottlenecks. Congestion phenomenon. Performance paradoxes.

Mini-Project proposed by student.

Learning Prerequisites

Required courses

A first course on probability

A first course on programming

Learning Outcomes

By the end of the course, the student must be able to:
• Estimate confidence intervals
• Design a simulation method
• Critique performance metrics and factors
• Organize a performance evaluation study
• Quantify performance
• Conduct a performance analysis
• Synthesize performance results
• Systematize factors and metrics
• Present results of a performance analysis

Transversal skills

• Use a work methodology appropriate to the task.
• Demonstrate the capacity for critical thinking

Teaching methods

Lectures + pencil and paper exercises + labs + miniproject

Expected student activities

Lectures

Paper and pencil exercises

Labs

Miniproject (last 4 weeks)

Tests every other week

Assessment methods

T = Average of best (n-1) tests done every other week except during miniproject period

E = grade at final exam (during exam session)

L = average of labs

M = miniproject grade

Final grade = 1/4 (T+E+L+M), rounded to the nearest half integer.

All grades except the final grade are not rounded.

Resources

No

Bibliography

• Performance Evaluation of Computer and Communication Systems, Le Boudec Jean-Yves, EPFL Press 2010
• also freely available online at perfeval.epfl.ch

Reference week

MoTuWeThFr
8-9
9-10
10-11
11-12
12-13
13-14
14-15
15-16
16-17
17-18
18-19
19-20
20-21
21-22
Under construction

Lecture
Exercise, TP
Project, other

legend

• Autumn semester
• Winter sessions
• Spring semester
• Summer sessions
• Lecture in French
• Lecture in English
• Lecture in German