Learning Theory

Language: English Credits: 4 ECTS Prerequisites:

• Analysis I, II, III
• Linear Algebra
• Machine learning
• Probability
• Algorithms (CS-250)

Here is a link to official coursebook information. Homework: Some homework will be graded. Grading: If you do not hand in your final exam your overall grade will be NA. Otherwise, your grade will be determined based on the following weighted average: 10 % for the Homework, 90 % for the Final Exam. For the graded homeworks,  you can discuss the homework with other people. But you have to write down your own solution and note on the first page the set of people that you discussed with.

Special Announcements

Lectures:

Lectures will be online on Zoom on Monday, from 8:15pm to 10:15pm. To access the session use https://epfl.zoom.us/j/95827964653 

If you have any questions during the lecture, use the chat to ask questions. The lectures will be recorded and the recordings are made available here https://mediaspace.epfl.ch/channel//29761

WhatsApp: there is also a WhatsApp chat. You can join using the following link: https://chat.whatsapp.com/IkgG5NpvsODDXSncJhJchk

Online exercise sessions:

Exercise sessions take place on Tuesdays, from 5pm to 7pm, using the same zoom link. We will solve one problem together at the beginning of the exercise session and in the remaining time we will answer your questions in smaller groups. The exercises sessions are not recorded. 

Graded homework

The graded homework are collected via this Moodle page. You can either write them by hand and scan or you can use latex: Latex template graded homework. If you cannot compile LaTeX on your own computer, EPFL is providing Overleaf Professional accounts for all students: Overleaf EPFL . With Overleaf you can write and compile LaTeX directly from your web browser. To use the provided template (.tex), you can create a new project and upload the .tex file. 

Final exam

The final exam is a 3 hour open-book on-campus exam (lecture notes, exercices, course material, but no electronic devices), held during the regular exam period. This exam will contribute 90% to the grade.

The exam is scheduled to take place on Thursday 24 June 2021 at 8:15-11:15, room INM200. 

Should circumstances not allow us to do this, we will replace these 90% with the following. You will be assigned a research paper and will be asked to read it and to write a 4 page latex-typed critical summary. You will be graded on this summary. You will be given a period of three weeks to complete this task. We expect that you spend one week on the assignment. You are allowed to discuss the paper with your colleagues. But the write-up must be yours. We will check for similarities.

Final exam 2019 with solutions 

Final exam 2020 with solutions

Problems 3 and 5.3 from 2019 exam and 5.1 from 2020 exam cover the material which was not given this year and would not appear in the exam.

Final exam 2021 with solutions

Q&A sessions prior to final exam

Q&A session will take place on Tuesday, 22 June at 14:00 on Zoom.

Topics

• PAC learning model (based on Chapters 2-7 in Understanding Machine Learning (UML) by Shalev-Shwartz and Ben David)
• Gradient descent (UML and notes by A. Montanari)
• Tensor decomposition (based on the review on Tensors Decompositions by Rabanser, Shchur and Günnemann). For more advanced material see also: Algorithmic Aspects of Machine Learning by Moitra

Detailed Schedule

Textbooks and notes:

• Understanding Machine Learning by Shalev-Shwartz and Ben David
• Bayesian Reasoning and Machine Learning by David Barber(Cambridge)
• Pattern recognition and Machine Learning by Christopher Bishop (Springer)
• Introduction to Tensor Decompositions and their Applications in Machine Learning (Ranbaser, Shchur, Gunneman)
• Probability on Graphs. Random processes on graph and lattices by Geoffrey Grimmett (Cambridge) [Chap 7]
• Neural Tangent Kernel references:

1. Original paper by Jakot et al.
2. An “easier to read” follow up paper by Arora et al.
3. A nice blog article by Rajat Vadiraj Dwaraknath