Student Projects

Background and Motivation:

The rapid growth of electric vehicles (EVs) is leading to a surge in the number of end-of-life (EOL) batteries, which require safe and efficient recycling processes. Disassembling battery packs is a complex and hazardous task, where robots can offer advantages in terms of precision, safety, and repeatability. However, existing robotic systems face challenges in adapting to the variability of battery pack designs.

At the Swiss Battery Technology Centre (SBTC), a reinforcement learning (RL) methodology is currently employed to train robots for various disassembly tasks, such as unscrewing components, using Nvidia Isaac Sim for simulation. The RL pipeline involves training robots to maximize cumulative rewards for effective disassembly in a simulated environment, followed by transferring these skills to real-world robotic setups.

A key challenge in this process is reward function design: for manipulation tasks, ground-truth rewards are often sparse (e.g., only rewarding task completion), leading to inefficient optimization. Manually crafting denser reward functions is time-consuming and frequently results in suboptimal learning. Furthermore, while helpful, automated reward learning methods rely heavily on costly human inputs, such as expert demonstrations or preference data. Recently, foundation models like large language models (LLMs) have shown promise in automating and enhancing reward function design, offering a cheaper and more scalable solution [1,2,3]. The goal of this project is to explore the potential of LLMs for iteratively improving reward functions in complex disassembly tasks.

Methodology:

1. Integration with Existing RL Pipeline

Integrate LLMs into SBTC’s RL pipeline to automate the generation and refinement of reward functions for tasks such as unscrewing EV battery components.

2. Iterative Reward Function Optimization

Utilize LLMs to iteratively improve reward functions based on feedback from simulation results. The LLM will analyze task outcomes and adjust the reward design to better align with desired behaviors, enhancing the robot’s performance.

3. Simulation-Based Training

Train robotic agents using the optimized reward functions within Nvidia Isaac Sim to evaluate improvements in learning speed and task efficiency.

4. Real-World Validation

Transfer the trained skills to real-world robotic setups. Assess the performance gains in disassembly tasks, focusing on precision, speed, and robustness.

Expected outcomes:

1. Automated RL training pipeline for developing new robotic manipulation skills with minimal human intervention.

2. Enhanced training efficiency due to optimized reward function design, leading to faster convergence and better generalization in real-world disassembly tasks.

3. Insights into using LLMs for continuous learning and adaptation in robotic systems, potentially extending beyond disassembly to other automation applications.

Requirements:

We look for motivated students with a strong background in machine learning and coding. We do have concrete ideas on how to tackle the above challenges, but we are always open for different suggestions. If you are interested, please send an email containing 1. one paragraph on your background and fit for the project, 2. your BS and MS transcripts to raphael.raetz@sipbb.ch, oezhan.oezen@sipbb.ch, and andreas.schlaginhaufen@epfl.ch.

References

[1] Ma, Yecheng Jason, et al. “Eureka: Human-level reward design via coding large language models.” arXiv preprint arXiv:2310.12931 (2023).

[2] Ma, Yecheng Jason, et al. “DrEureka: Language Model Guided Sim-To-Real Transfer.” arXiv preprint arXiv:2406.01967 (2024).

[3] Sun, Yuan, et al. “Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF.” Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2024.

Professor(s)

Maryam Kamgarpour (Recherche en systèmes), Andreas Schlaginhaufen

Administration

Barbara Marie-Louise Frédérique Schenkel

External

Raphael Rätz, Özhan Özen, raphael.raetz@sipbb.ch, oezhan.oezen@sipbb.ch

Site

https://www.epfl.ch/labs/sycamore/student-projects/

The EPFL babyfoot is under continuous improvement.

While the babyfoot can easily intercept the ball and kick toward the opponent goal with success, it can also juggle and shoot if the ball is still.

The difficulty is to capture the ball, make a pass and chain precise actions. This project aims at improving the ball handling while it is moving to permit capture, pass and juggling, and then shoot toward opponents goal.

Students suggested improvements are also welcome.

The babyfoot is programmed using LabVIEW, knowledge of this language is prerequisite for strategy related projects.

Professor(s)

Christophe Salzmann (Laboratoire d’automatique 3)

Site

https://www.epfl.ch/labs/la/pi/babyfoot/

Motivation:

Although conventional controllers might be mathematically optimal with respect to a predefined objective, they often fail to produce driving trajectories that align with the preferences or expectations of human passengers. An alternative approach involves learning a control policy from human “expert” demonstrations so as to imitate human driving behavior. However, directly fitting a trajectory to observed data-also known as behavioral cloning-often suffers from instability and poor generalization to unseen states. To address this, a simulator can be leveraged to ensure that the learned controller generates trajectories that remain close to the expert data throughout.

Description:

In this project, we would like to apply imitation learning methods to an existing dataset of human-driven trajectories. A behavioral cloning baseline is already available. The goal is to compare this baseline to a closed-loop approach that utilizes an existing simulator. In this setup, the policy could be parameterized by a neural network or a model predictive control (MPC) policy. Ideally, after development in simulation, the project leads to the successful deployment of the controller on our ready-to-use real-world mini race car system.

Skills needed

– Strong background in Machine Learning and the relevant software tools (Python, PyTorch).

– Qualitative understanding of car dynamics or similar mobile robots.

– Significant experience in coding projects.

– Familiarity with ROS/ROS2 (Robot Operating System).

– Familiarity with C++ is a plus (for real-time deployment).

Comment

Directly contact johannes.waibel@epfl.ch and andreas.schlaginhaufen@epfl.ch if interested.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Johannes Christian Karl Waibel

Administration

Barbara Marie-Louise Frédérique Schenkel

The environmental and economic significance of wind power in the 21st century is mainstream knowledge The aim of the project is the implementation of a robust Model Predictive Control (MPC) scheme for Wind Farms (WF) capable of dealing with noise uncertainties inherently present in the. At the same time, the controller should fit the required power, maximize the performance and minimize the mechanical forces acting on the turbines. The student will leverage the new results in Wasserstein Tube MPC [1] which is an extension of the well-known Tube MPC. This new approach is less conservative than the standard techniques used in robust control which leads to better performances of the closed-loop.

Skills needed:

Coding proficiency in Matlab (!!!)

Understanding of MPC (!!)

MPT3 toolbox knowledge is a plus (!)

Mathematical skills are a plus (!)

Professor(s)

Giancarlo Ferrari Trecate, Riccardo Cescon

Administration

Barbara Marie-Louise Frédérique Schenkel

ROJECT SUMMARY

Adaptive optics (AO) systems are critical in overcoming atmospheric distortions in ground-based telescopes, enabling sharper and more detailed astronomical observations. This project focuses on designing and synthesising a robust, real-time controller for an AO system. By leveraging advanced control theory, the project aims to significantly enhance the performance of AO systems, ensuring reliable correction of wavefront distortions and improving the quality of astronomical images.

BACKGROUND AND MOTIVATION

Telescopes observing through Earth’s atmosphere suffer from distortions caused by turbulent air layers, which degrade image resolution. Adaptive optics counteract this by using deformable mirrors and wavefront sensors to correct for distortions in real-time.

Current AO systems, while effective, face limitations in:

* Handling dynamic, unknown atmospheric conditions.

* Scalability for large telescopes and next-generation systems.

By synthesising a controller tailored to these challenges, this project seeks to push the boundaries of AO performance, paving the way for discoveries in astronomy and astrophysics.

OBJECTIVES

1. Design and develop a robust controller that addresses dynamic atmospheric variations.

2. Validate performance through simulation (and hardware) implementation using an AO testbed.

3. Ensure scalability and adaptability of the controller for different telescopes and operational conditions.

REQUIREMENTS

The project demands a solid academic foundation in control courses, particularly in ‘Advanced Control Systems.’ Proficiency in the frequency domain approach and robust control techniques is especially critical.

Professor(s)

Alireza Karimi, Vaibhav Gupta

Administration

Barbara Marie-Louise Frédérique Schenkel

External

Department of Astronomy, UNIGE, Isaac Dinis, isaac.dinis@unige.ch

Site

ddmac.epfl.ch, la.epfl.ch

Professor(s)

Alireza Karimi, Mahdi Mehdikhani Sarvjahany

Description:

Deep neural network (DNN) controllers have demonstrated remarkable success in controlling complex and nonlinear systems. These controllers are trained using data, eliminating the need to accurately model the system, which is often challenging due to the system’s complexity and inherent uncertainties. However, the stability and performance of DNN controllers when deployed in real-world remain significant concerns. While DNN controllers perform well within their training datasets, their behavior in scenarios outside the training data is not fully understood.

This project aims to design and evaluate DNN controllers for heating systems that incorporate heat storage. The heat storage functions as a battery: it heats (charges) when electricity generation is high (e.g., from renewable sources) and releases heat (discharges) when the heating demand is higher. The primary goal is to reduce heating costs while meeting heating demands. Ensuring the stability and performance of the designed controllers is crucial for safe operation and cost efficiency in real-world applications. To achieve this, we will employ the controller design approach introduced in [1], which provides stability and performance guarantees.

Comment

Main tasks:

Understand the fundamentals and dynamics of heating systems, including the role of heat storage (see, e.g., [2]).

Tailor DNN controllers for heating systems with integrated heat storage.

Follow the methodology outlined in [1] to train DNN controllers that ensure stability.

Established performance guarantees for the developed controller as per the approach in [1].

References:

[1] Mahrokh Ghoddousi Boroujeni, Clara Lucía Galimberti, Andreas Krause, & Giancarlo Ferrari-Trecate. (2024). A PAC-Bayesian Framework for Optimal Control with Stability Guarantees.

[2] Bünning, F., Warrington, J., Heer, P., Smith, R., & Lygeros, J. (2022). Robust MPC with data-driven demand forecasting for frequency regulation with heat pumps. Control Engineering Practice, 122, 105101.

Professor(s)

Giancarlo Ferrari Trecate, Christophe Salzmann

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

Description:

Deep neural network (DNN) controllers have demonstrated remarkable success in controlling complex and nonlinear systems. These controllers are trained using data, eliminating the need to accurately model the system, which is often challenging due to the system’s complexity and inherent uncertainties. However, the stability and performance of DNN controllers when deployed in real-world remain significant concerns. While DNN controllers perform well within their training datasets, their behavior in scenarios outside the training data is not fully understood.

This project aims to design and evaluate DNN controllers for heating systems that incorporate heat storage. The heat storage functions as a battery: it heats (charges) when electricity generation is high (e.g., from renewable sources) and releases heat (discharges) when the heating demand is higher. The primary goal is to reduce heating costs while meeting heating demands. Ensuring the stability and performance of the designed controllers is crucial for safe operation and cost efficiency in real-world applications. To achieve this, we will employ the controller design approach introduced in [1], which provides stability and performance guarantees.

Comment

Contacts:

mahrokh.ghoddousiboroujeni@epfl.ch

nicolas.kirsch@epfl.ch

Main tasks:

Understand the fundamentals and dynamics of heating systems, including the role of heat storage (see, e.g., [2]).

Tailor DNN controllers for heating systems with integrated heat storage.

Follow the methodology outlined in [1] to train DNN controllers that ensure stability.

Established performance guarantees for the developed controller as per the approach in [1].

References:

[1] Mahrokh Ghoddousi Boroujeni, Clara Lucía Galimberti, Andreas Krause, & Giancarlo Ferrari-Trecate. (2024). A PAC-Bayesian Framework for Optimal Control with Stability Guarantees.

[2] Bünning, F., Warrington, J., Heer, P., Smith, R., & Lygeros, J. (2022). Robust MPC with data-driven demand forecasting for frequency regulation with heat pumps. Control Engineering Practice, 122, 105101.

Professor(s)

Giancarlo Ferrari Trecate

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

Nonlinear Model Predictive Control (NMPC) has achieved great success in legged locomotion in the past decade, where the numerical solver plays a key role.

In this project, we aim to deploy LAopt, an NMPC solver developed at the EPFL Automatic Control Laboratory, onto a Unitree A1 robot to achieve some basic locomotion tasks on flat ground.

The project will contain the following working packages:

1. Get familiar with the current state-of-the-art MPC formulation on quadrupedal locomotion.

2. Implement the MPC controller in C++ using LAopt and the robot dynamics library Pinocchio. Test the performance in simulation.

3. Deploy and test on a Unitree A1 robot (if time permits)

Reference:

R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter, “Perceptive Locomotion Through Nonlinear Model-Predictive Control,” IEEE Transactions on Robotics, vol. 39, no. 5, pp. 3402-3421, Sep. 2023, doi: 10.1109/tro.2023.3275384.

Comment

Skills needed

– Significant experience in C++ and ROS/ROS2

– Good understanding of Model Predictive Control and Rigid Body Dynamics

– Familiarity with robot dynamics libraries (Pinocchio, RBDL) is a plus

– Familiarity with numerical optimization is a plus

Contact:

Fenglong Song – fenglong.song@epfl.ch

Roland Schwan – roland.schwan@epfl.ch

Shaohui Yang – shaohui.yang@epfl.ch

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3)

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

The problem of optimal portfolio selection in a market which exhibits return predictability, price impact and partial observability can be modeled as a linear-quadratic Gaussian (LQG) optimal control problem [1]. However, financial modeling generally involves high uncertainty. Apart from the noise being stochastic, the probability distribution generating the noise may be unknown. Distributionally Robust Optimization (DRO) addresses this challenge by considering the worst-case noise distribution within an ambiguity set of plausible distributions. For our purposes, we will consider ambiguity sets defined using the Wasserstein metric on the space of probability distributions. This semester project aims to combine the DRO and LQG approaches for the optimal portfolio selection problem, modeling and solving it as a distributionally robust linear-quadratic Gaussian (DR-LQG) optimal control problem [2].

Main tasks:

1. Formulate the optimal portfolio selection problem under uncertain noise distributions as a DR-LQG problem by combining the results of [1] and [2].

2. Solve the DR-LQG problem numerically and compare the solution with the plain LQG approach in [1], as well as other state-of-the-art approaches for portfolio selection (e.g., Markowitz).

Comment

Contact:

Jakob Nylöf

jakob.nylof@epfl.ch

Prerequisites:

– Control theory and optimization

– Programming in Python

– Good mathematical skills

References:

[1] Marc Abeille, Emmanuel Sérié, Alessandro Lazaric, Xavier Brokmann. (2016). LQG for Portfolio Optimization.

[2] Bahar Taskesen, Dan A. Iancu, Cagil Kocyigit, Daniel Kuhn. (2023). Distributionally Robust Linear Quadratic Control.

Professor(s)

Giancarlo Ferrari Trecate, Lars Jakob Nylöf

Administration

Barbara Marie-Louise Frédérique Schenkel

Motivation:

Graphical Processing Units (GPUs) are advancing at a significantly faster pace than Central Processing Units (CPUs). GPUs excel at handling small, repetitive tasks in parallel, making them well-suited for computationally intensive applications. In Model Predictive Control (MPC), solving multiple instances of a problem arises in scenarios such as:

1. When high-quality solutions are required, multiple initial guesses are needed to bypass local minima.

2 When addressing mixed-integer problems with numerous combinations to explore.

Recent efforts in the robotics community have initiated the shift of MPC solvers from CPU to GPU platforms, as demonstrated by Adabag et al. (2024) in their work on real-time nonlinear MPC on GPUs. However, existing approaches primarily focus on single-problem instances. This project aims to develop GPU-based MPC solvers optimized for solving multiple instances simultaneously, in collaboration with Columbia University.

Reference:

Adabag, Emre, et al. “MPCGPU: Real-time nonlinear model predictive control through preconditioned conjugate gradient on the GPU.” 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.

Project Timeline:

Literature Review:

1. Study existing GPU-based methods for MPC solvers;

2. Focus on iterative linear solvers, interior point QP solvers, and initial guess generation strategy;

3. Conduct preliminary trials using MATLAB.

Implementation Enhancement:

1. Optimize the existing MPCGPU solver implemented in C/C++ with CUDA.

Performance Comparison:

1. Benchmark the optimized solver against existing CPU and GPU-based Linear MPC solvers.

Expected Outcomes:

1. Development and potential publication of advanced GPU-optimized MPC solvers.

2. Detailed performance analysis comparing CPU and GPU-based solvers.

Requirements:

1. Proficiency in C/C++, with experience in CUDA programming or a willingness to learn.

2. Strong foundation in numerical linear algebra and convex optimization, particularly linear system solvers, condition numbers, and the interior point method.

3. Basic understanding of Model Predictive Control (MPC).

Comment

If you are interested, please send your CV to shaohui.yang@epfl, specifying your interest in this project.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Shaohui Yang

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

Motivation:

Although they might be optimal with respect to some mathematical objective, conventional controllers might not result in driving trajectories that are preferred or expected by human passengers. One approach for a different family of controllers is to train a Machine Learning model from human ‘expert’ demonstrations, such that the resulting controller seems to act similarly to a human driver.

Description:

In this project, we would like to learn a Neural Network controller from a large existing dataset of human-driven trajectories. This includes implementing a baseline Imitation Learning method first, and a more advanced method second. Arising challenges are typically related to the stability of the learnt controller and its behavior outside the training data distribution.

Ideally, after development in simulation, the project leads to the successful deployment of the controller on our ready-to-use real-world mini race car system.

Skills needed

– Strong background in Machine Learning and the relevant software tools (Python, PyTorch)

– Qualitative understanding of car dynamics or similar mobile robots

– Significant experience in coding projects

– Familiarity with ROS/ROS2 (Robot Operating System) is a plus

– Familiarity with C++ is a plus (for real-time deployment)

Comment

Directly contact johannes.waibel@epfl.ch and andreas.schlaginhaufen@epfl.ch if interested.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Johannes Christian Karl Waibel

Administration

Barbara Marie-Louise Frédérique Schenkel

The goal of the project is to study an application of the Koopman operator based control approach. For autonomous systems, Koopman operator theory shows that a nonlinear system can be represented as a linear system in a higher dimensional (possibly infinite dimensional) space. Thus, by lifting a nonlinear system to a higher dimensional space this approach enables the use of tools from linear systems theory for studying nonlinear systems. By using the Koopman operator approach, our goal is to obtain a linear system representation in a high dimensional space and employ linear control design techniques to control the actually nonlinear system.

The lifted system representation will be obtained in a data-driven fashion. As a result of this as well as further structural restrictions, the lifted representation never exactly captures the original system dynamics. Thus, the project will also aim for bounding these errors such that guarantees for the true closed-loop system can be provided by using robust control tools.

For now a pendulum system is considered as the application.

Skills needed:

-Advanced control systems (familiarity with frequency domain approach and robust control)

-Matlab/Simulink, LabVIEW

Comment

Contact: mert.eyuboglu@epfl.ch

Professor(s)

Alireza Karimi

Administration

Mert Eyuboglu

Site

ddmac.epfl.ch, la.epfl.ch

The goal of this project is to apply the Koopman operator-based modeling and control approach to a 3DOF gyroscope system. Using the Koopman operator, we aim to obtain a linear system representation in a high-dimensional space, enabling the application of linear control design techniques to an inherently nonlinear system. The lifted system representation will be identified in a data-driven manner and validated on the real system. Once this model is validated, a linear controller will be designed in the lifted space and subsequently tested on the actual system.

Skills needed:

-System identification

-Matlab/Simulink, LabVIEW

Comment

Contact: mert.eyuboglu@epfl.ch, vishnu.varadan@epfl.ch

Professor(s)

Alireza Karimi

Administration

Mert Eyuboglu

Site

ddmac.epfl.ch, la.epfl.ch

Motivation: Shared decision-making is at the heart of any multi-agent system. Each agent independently aims to maximize its reward which depends on the decisions of all agents. The outcome of

such independent decision-making, however, is often inefficient in terms of social welfare and may violate other central objectives such as fairness. In traffic routing, for example, each driver aims to minimize its travel time independently which can result in traffic jams. Traffic routing under a central authority, however, can lead to a reduction in the aggregate travel time of all drivers. The field of mechanism design is concerned with how a central authority can incentivize independent decision-making towards an outcome that is aligned with the social welfare, possibly by incurring a cost on each agent. A key challenge is that the social welfare function can be complex and taking into account several possibly conflicting objectives, and may thus be a priori unknown. To address this, this master project aims to incorporate feedback from the central authority to learn a social welfare function that is aligned with the central authority’s preferences.

Comment

Requirements: We seek for motivated students with a strong mathematical, or computer science background. We do have some concrete ideas on how to tackle the above challenges, but we are always open for different suggestions. If you are interested, please send an email containing 1. one paragraph on your background and fit for the project, 2. your BS and MS transcripts to anna.maddux@epfl.ch and andreas.schlaginhaufen@epfl.ch.

Professor(s)

Maryam Kamgarpour (Recherche en systèmes), Anna Maria Maddux

Administration

Sandra de Best

Site

https://www.epfl.ch/labs/sycamore/student-projects/

Description: The proposed project aims to explore and evaluate the application of meta-learning techniques in control systems. Meta-learning focuses on enabling models to adapt quickly to new tasks with limited data by leveraging existing knowledge. The student will assess the performance of existing meta-learning algorithms, such as Model-Agnostic Meta-Learning (MAML), in control system tasks such as optimal control. Simulations will be performed on the Franka Robot, bridging theory and real-world applications. By integrating these approaches with control-specific architectures and evaluating their efficacy in simulation, the project seeks to identify the challenges and opportunities of meta-learning for dynamic system adaptation. This project will provide insights into the potential of meta-learning for creating flexible, efficient, and generalizable control strategies.

Main tasks:

-Familiarize with the application in optimal control

-Learn about meta-learning approaches

-Implement and evaluate on simulations with the Franka Robot

Comment

Contact:

mahrokh.ghoddousiboroujeni@epfl.ch

laura.meroi@epfl.ch

Professor(s)

Giancarlo Ferrari Trecate, Christophe Salzmann

Administration

Barbara Marie-Louise Frédérique Schenkel

1. Overview

This project focuses on developing trajectory planning solutions for autonomous systems in interactive environments. The goal is to create algorithms enabling each robot to safely reach its destination while navigating complex scenarios, such as multiple autonomous cars at non-signalized intersections or multi-robot systems requiring formation control and coordination. Key challenges include collision avoidance with other agents and obstacles. The methodologies will be implemented and tested on an Nvidia Jetbot multi-robot testbed.

2. Responsibilities

– Literature review of existing trajectory planning methods within the frameworks of stochastic optimal control and game theory.

– Implementing the chosen control algorithms on the robotic testbed.

– Evaluating the algorithms for optimality, safety, and computational efficiency.

– Developing new algorithmic approaches for multi-robot trajectory planning based on the needs of the problem.

3. Requirements

– Familiarity with ROS/ROS2 and independence in trouble-shooting.

– Proficiency in Python for algorithm development and system integration.

– The project requires strong academic performance in control courses including model predictive control (MPC).

Comment

Note: To apply, send an email to kai.ren@epfl.ch including 1. one paragraph on your background and fit for the project, 2. your BS and MS transcripts. This project will be supervised by Prof. Maryam Kamgarpour and Kai Ren (kai.ren@epfl.ch).

Professor(s)

Maryam Kamgarpour (Recherche en systèmes), Kai Ren

Administration

Sandra de Best

Neural network controllers are becoming increasingly popular because of their ability to control complex and nonlinear systems. They manage to induce advanced behavior unmatched by standard control approaches. However, they often lack guarantees about stability and steady-state behavior, which is a major concern for real world applications. The methodology developed in [1] successfully profits from neural network controllers while also guaranteeing closed loop properties for stabilization problems.

The objective of this project is to generalize this approach to reference tracking problem. The developed controller should optimize the transient phase while preserving good steady state tracking performance. This will be then extended to networked systems with distributed controllers, building on the work done in [2]. Furthermore, the designed controller will be tested in simulations for distributed robots.

Main tasks:

– Familiarize yourself with the state-of-the-art approaches to neural network control with stability guarantees for simple and networked system developed in [1] and [2].

– Design an initial reference tracking controller for single agent systems and test it in simulations

– (Optional) Look at extending the theory to networked systems

– Design the distributed reference tracking controller for networked system and test it in a multi robot environment.

Skills needed:

– Sound mathematical and control background

– Coding in Python, especially in Pytorch

– Literature understanding and curiosity

[1]: L. Furieri, C. L. Galimberti and G. Ferrari-Trecate, “Learning to Boost the Performance of Stable Nonlinear Systems,” in IEEE Open Journal of Control Systems, vol. 3, pp. 342-357, 2024

[2]: D. Saccani, L.Massai, L. Furieri and G. Ferrari-Trecate, “Optimal distributed control with stability guarantees by training a network of neural closed-loop maps” on ArXiv DOI: https://doi.org/10.48550/arXiv.2404.02820

Comment

Directly contact nicolas.kirsch@epfl.ch if interested.

Professor(s)

Giancarlo Ferrari Trecate, Nicolas Léo Pierre Kirsch

Administration

Barbara Marie-Louise Frédérique Schenkel

Objective:

This project focuses on designing and synthesizing a state estimator for large and distributed systems, utilizing state-of-the-art optimization techniques. You will explore advanced control methods to develop a reliable estimator capable of efficiently managing the complexities of distributed systems, even in the presence of unknown and unforeseen disturbances. The approach will leverage recent advancements in robust control theory [1,2] and incorporate them into a state estimation framework, as demonstrated in [3]. Success in this project will require you to integrate knowledge from control theory, computer science, and optimization.

Project Overview:

State estimation is crucial in modern control systems, particularly in large and distributed networks where each sub-system must rely only on local and neighbours information. Your task will be to:

1. Conduct a Literature Review: Begin by surveying the existing body of knowledge on state estimation techniques and minimum regret control [1-3].

2. Design and Synthesis: Based on your literature review, design a state estimator tailored to a large, distributed system.

3. Simulation: Implement and test your state estimator using either Python or MATLAB. Simulate various scenarios to assess the performance of your design under different conditions.

Skills Required:

* Control Theory: A solid understanding of control systems and state estimation techniques is essential.

* Computer Science: Proficiency in programming, particularly in Python or MATLAB, is necessary for the simulation part of the project.

* Optimization: Knowledge of optimization methods will help in refining your state estimator for better performance and efficiency.

References:

[1]: Martin, Andrea, et al. “Safe control with minimal regret.” Learning for Dynamics and Control Conference. PMLR, 2022.

[2]: Martinelli, Daniele, et al. “Closing the gap to quadratic invariance: a regret minimization approach to optimal distributed control.” 2024 European Control Conference (ECC). IEEE, 2024.

[3]: Brouillon, Jean-Sébastien, Florian Dörfler, and Giancarlo Ferrari Trecate. “Minimal regret state estimation of time-varying systems.” IFAC-PapersOnLine 56.2 (2023): 2595-2600.

Comment

For more information, about the project, feel free to contact us by mail: daniele.martinelli@epfl.ch

For a better-formatted description of the project, please refer to the attached pdf.

Professor(s)

Giancarlo Ferrari Trecate, Daniele Martinelli

Administration

Barbara Marie-Louise Frédérique Schenkel

External

daniele.martinelli@epfl.ch

During the past few years, policy gradient methods have gained renewed attention within the control community, primarily due to their global convergence properties for certain classical control problems, such as $\mathcal{H}_2$ and $\mathcal{H}_\infty$ optimal control. However, the characteristics of policy gradient methods in the context of robust control with data-dependent uncertainty remain an open question.

This project focuses on the design and analysis of policy gradient algorithms for data-driven robust control. Given a set of noisy offline input-output data, and under assumptions on the boundedness of noise, the goal is to propose policy gradient algorithms minimizing worst-case closed-loop performance. Moreover, the properties of the proposed algorithms, such as global optimality and convergence rate, will be investigated.

Comment

Required knowledge:

1. Control theory

2. Convex optimization

3. Numerical analysis

References:

[1] Maryam Fazel, et al. “Global convergence of policy gradient methods for the linear quadratic regulator.” International conference on machine learning. PMLR, 2018.

[2] Kaiqing Zhang, Bin Hu, and Tamer Basar. “Policy Optimization for $\mathcal{H}_2$ Linear Control with $\mathcal{H}_\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence.” Learning for Dynamics and Control. PMLR, 2020.

[3] Yang Zheng, Chih-fan Pai, and Yujie Tang. “Benign nonconvex landscapes in optimal and robust control, Part I: Global optimality.” arXiv preprint arXiv:2312.15332 (2023).

Professor(s)

Alireza Karimi, Zhaoming Qin

Administration

Barbara Marie-Louise Frédérique Schenkel

Motivation:

Finding the right numerical values for tuning a controller can be a difficult task for a human operator. Choosing between two options (“Which one do I like better?”) can, however, be a more intuitive approach. Preferential Bayesian Optimization (BO) exploits this by optimizing an unknown cost function from such choice samples.

Description:

In this project, we would like to apply Preferential Bayesian Optimization to a mini racecar system. We will use the existing Python library POP-BO to suggest the user two trajectories of the racecar. Based on the human preference, the algorithm will discover and minimize the hidden cost function that the human has (implicitly) in mind. We will first use this for a simple tuning case (e.g. ‘aggressiveness’ of driving), and then extend it for more sophisticated test cases.

Ideally, after development in simulation, the project leads to the successful deployment of the controller on our ready-to-use real-world mini race car system.

Skills needed

– Background in Machine Learning (Gaussian process), and the relevant software tools (Python, PyTorch, ideally GPy).

– Qualitative understanding of car dynamics or similar mobile robots.

– Significant experience in coding projects.

– Familiarity with ROS/ROS2 (Robot Operating System).

– Familiarity with C++ is a plus (for real-time deployment).

Comment

Directly contact wenbin.wang@epfl.ch and johannes.waibel@epfl.ch if interested.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Wenbin Wang

Administration

Barbara Marie-Louise Frédérique Schenkel

Energy consumption in residential buildings constitutes a significant portion of total energy use. Reducing energy consumption in buildings while maintaining thermal comfort for occupants is critical, which requires the development of accurate thermal comfort models. A common approach to constructing such models is Bayesian optimization. While effective at identifying optimal points, traditional Bayesian optimization methods often overlook the dynamic behavior of systems, leading to suboptimal efficiency.

This project aims to address this limitation by performing Bayesian optimization in a receding horizon fashion. Unlike the classical approach, which assumes that the next proposed point is fully reached, this method accounts for system dynamics and optimizes performance without requiring full convergence to the proposed point. The approach will be evaluated through simulations conducted on the BOPTEST building simulation platform.

Comment

Requirements

-Proficient in Python

-Knowledge in MPC

-Knowledge/strong interest in Bayesian optimization/building simulation

Contact: wenbin.wang@epfl.ch

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3)

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

The main goal is to solve Constrained Markov Decision Processes (CMDPs) while ensuring constraint satisfaction during learning. In Paper 2, the authors introduce the Constrained Policy Optimization (CPO) algorithm, which guarantees constraint satisfaction if the problem (10) can be solved exactly. However, without knowing the MDP transition dynamics, we can only solve it stochastically, where safety may no longer be guaranteed.

The project idea is to apply the concept from Paper 1 to CMDPs using a local convex approximation. By incorporating a safety filter or conservatively learning the constraint, our aim is to consistently satisfy constraints. Additionally, we will seek theoretical guarantees for optimality.

Professor(s)

Tingting Ni (Recherche en systèmes), Maryam Kamgarpour

Administration

Maryam Kamgarpour

Non-convex optimization arises naturally in many robotics applications, e.g., collision avoidance in autonomous driving. Gradient-based methods can be easily trapped in local minimas and even fail to find a solution. Sampling-based method can be less sensitive to local minimas.

In this project, we mainly consider developing effective methods for collision avoidance in mini race car systems. We would like to explore the capabilities of using sampling-based method to escape from local minimas. A novel method called Stein Variational Gradient Descent (SVGD) will be use to directly find the solution of the underlying Optimal Control Problem, or provide a good initial guess for the State-of-the-Art gradient-based MPC solvers.

Ideally, after successful development in simulation, the project leads to the deployment of the controller on our ready-to-use real-world mini race car system.

Comment

Skills needed

– Qualitative understanding of car dynamics or similar mobile robots

– Significant experience in coding projects (e.g., using Python or Matlab)

– Familiarity with Bayesian Inference is a plus

– Familiarity with ROS/ROS2 (Robot Operating System) is a plus

– Familiarity with C++ is a plus (for real-time deployment)

Please send enquiries to : Fenglong Song (fenglong.song@epfl.ch) Johannes Waibel (johannes.waibel@epfl.ch)

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3)

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

Motivation:

Linear Model Predictive Control (MPC) problems can be efficiently solved by formulating them as Quadratic Programming (QP) problems, utilizing the structure of the resulting Karush-Kuhn-Tucker (KKT) system. For linear time-varying (LTV) systems, the left-hand side of the KKT system forms a block tri-diagonal matrix, a property extensively exploited in prior research. In linear time-invariant (LTI) systems, where the dynamics pair (A, B) remains constant, additional sparsity and computational advantages can be achieved through transformations. These modifications enhance the structure of the KKT system, facilitating easier and more efficient computation.

Project Timeline:

1. Literature Review: Familiarize with existing methods for transforming linear control systems.

2. Implementation Enhancement: Refine and optimize the current Sparse Linear MPC solver implemented in C++ with Eigen as the linear algebra backend.

3. Performance Comparison: Benchmark the improved solver against a variety of existing Linear MPC solvers.

4. Extension: If time permits, extend the implementation to support robust and stochastic Linear MPC.

Expected Outcomes:

1. Development and potential publication of advanced (robust/stochastic) MPC solvers tailored for LTI systems.

2. Comprehensive performance comparisons with existing solvers.

Requirements:

1. Proficiency in C++.

2. Strong background in numerical linear algebra and scientific computation, including matrix factorizations, condition numbers, and error analysis.

3. Basic knowledge of Model Predictive Control (MPC) and Convex Optimization, particularly QP formulation.

Comment

If interested, please send your CV to shaohui.yang@epfl and mention your specific interest in the project.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Shaohui Yang

Administration

Barbara Marie-Louise Frédérique Schenkel

Site

https://www.epfl.ch/labs/la/pi/

The aim of this semester project is to derive a dynamical model of the cooling system of a large business center. This system consists of several interconnected subsystems that interact through an extensive pipe network, and includes variable user demands as well as third-parties cooling components. These complexities make traditional modeling methods inadequate, requiring advanced approaches such as neural networks for accurate system identification. Interconnected neural networks allow to preserve the sparsity of the system, leading to the identification of each subsystem as well as the overall interconnected system. A precise model of each subsystem is fundamental for efficient monitoring and control, ultimately leading to energy savings and improved user comfort. Available measurements data from the real system will be used to train the designed interconnected neural network architecture.

Required knowledge and tools:

* Control theory

* Machine learning

* Simulink and Matlab

* Python and relevant libraries (PyTorch) for neural network implementation.

Reference papers:

[1] Terzi, E., Fagiano, L., Farina, M., & Scattolini, R. (2020). Structured modelling from data and optimal control of the cooling system of a large business center. Journal of Building Engineering, 28, 101043.

https://re.public.polimi.it/retrieve/0187b319-a62c-4570-b738-772c4d56d5f3/11311-1141455_Scattolini.pdf

[2] Terzi, E., Bonetti, T., Saccani, D., Farina, M., Fagiano, L., & Scattolini, R. (2020). Learning-based predictive control of the cooling system of a large business centre. Control Engineering Practice, 97, 104348. https://fagiano.faculty.polimi.it/docs/papers/BuildingMPC_ConEngPrac_2020.pdf

[3] Massai, L., Saccani, D., Furieri, L., & Ferrari-Trecate, G. (2023). Unconstrained learning of networked nonlinear systems via free parametrization of stable interconnected operators. arXiv preprint arXiv:2311.13967.

https://arxiv.org/pdf/2311.13967

[4] Saccani, D., & Bonetti, T. (2018). Modeling and learning-based hybrid predictive control with recurrent neural networks of the cooling station of a large business center.

Comment

Contact: laura.meroi@epfl.ch

Professor(s)

Giancarlo Ferrari Trecate, Laura Meroi

Administration

Barbara Marie-Louise Frédérique Schenkel