Master Thesis Projects are started once the complete master program is finished and all the credits have been obtained.
Projects for SSC and SIN students should last 4 months at the EPFL or 6 months in the industry or in another University.
Master Thesis Projects must be done individually.
Master Thesis Projects are worth 30 credits.
Students must have the approval of the Professor in charge of the laboratory before registering for the given project.
List of Projects – Spring 2025
Merits of curiosity in the face of extrinsic rewards
Curiosity is a crucial element of behavior that allows humans and animals to flexibly adapt to unknown and changing environments [1, 2, 3]. Exactly why and how we are curious in different situations, however, remains unclear. One hypothesis is that curiosity has developed throughout evolution as a way to guide behavior and maximize rewards in environments where extrinsic rewards, such as food and other resources for survival, are only sparsely and irregularly available [4]. But how can we make this hypothesis quantifiable and testable? In recent work, we developed an algorithmic framework that quantifies the effect the environment structure on the merits of curiosity during reward-free exploration [5]. Specifically, we found that the usefulness of different aspects of curiosity depends on the state and transition structure of the environment, and that, in the absence of extrinsic rewards, the two main drivers of curiosity are novelty and information gain [5]. In this project, we will extend our approach to quantify how the presence of extrinsic rewards affects the benefits of different aspects of curiosity.
Good programming skills in julia or python are required. Interested students should send their application, including CV and grades in relevant classes, to [email protected] and [email protected].
[1] Poli et al. (2024) Curiosity and the dynamics of optimal exploration. Trends in Cognitive Sciences, 28(5):441–453.
[2] Monosov (2024) Curiosity: primate neural circuits for novelty and information seeking. Nature Reviews Neuroscience, (25):195–208.
[3] Modirshanechi et al. Curiosity-driven exploration: foundations in neuroscience and computational modeling (2023) Trends in Neurosciences, 46(12):1054–1066.
[4] Singh et al. (2010) Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Au-
tonomous Mental Development, 2(2):70–82.
[5] Gruaz et al (2024). Merits of curiosity: a simulation study. (https://osf.io/evm9n/download)
Identifying monosynaptic connection using deep learning method (also possible as a Semester project)
Recently, we developed an approach to measure synaptic connectivity in vivo, training a deep convolutional network to reliably identify monosynaptic connections from the spike-time cross-correlograms of millions of single-unit pairs.
The benchmark results on both the experimental recordings and synthetic datasets indicate that the method is very promising.
In this project, you will learn
- methods about synaptic connectivity inference,
- and how to simulate a large network of spiking neurons for benchmarking,
while
- trying to improve the performance of the current method further. (I already have a few ideas for you to start with.)
The project is perfect for a student with a decent Python programming background and who wants to study applied machine learning problems in neuroscience.
The minimum requirements for the student are to write clear codes and be interested in this project.
Interested candidates, please send their application to [email protected]. If you have any questions, also feel free to contact me.
List of Projects – Autumn 2024
Modeling the impact of stimulus similarities on novelty perception using deep learning for latent representations (taken)
Novelty is an intrinsic motivational signal that guides the behavior of humans, animals and artificial agents in the face of unfamiliar stimuli and environments. But how can an (biological or artificial) agent determine whether a stimulus is novel or not? In the lab, we recently showed how algorithmic models of novelty detection in the brain [1,2] can be extended to continuous environments and stimulus spaces with similarity structures [3]. However, our current model relies on existing stimulus representations that are either constructed based on experimental knowledge or derived from pre-trained, brain-like deep networks. In machine learning, on the other hand, novelty is computed using neural networks that are trained end-to-end to estimate the stimulus novelty [4,5], which makes it hard to understand how the structure of the stimulus space influences novelty computation.
In this project, we will take a hybrid approach to model how novelty can be computed in naturalistic stimulus spaces, and combine deep learning with algorithmic models of novelty computation. We will investigate how algorithmic novelty signals can be used to instruct representational learning and how these representational changes in turn affect the computation of novelty. Finally, we will compare the novelty signals predicted by our model to state-of-the-art algorithmic and machine-learning models of novelty computation.
Good programming skills in python and prior experience with deep learning and pytorch are required. Interested students should send their application, including CV and grades in relevant classes, to [email protected].
References
[1] Xu et al., Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making. PLOS Comp. Biol. (2021)
[2] Modirshanechi et al., Surprise and novelty in the brain. Curr. Op. Neurobiol. (2023)
[3] Becker et al., Representational similarity modulates neural and behavioral signatures of novelty. biorxiv (2024)
[4] Bellemare et al., Unifying count-based exploration and intrinsic motivation. NeurIPS (2016)
[5] Ostrovski et al., Count-based exploration with neural density models. PMLR (2017)
List of Projects – Spring 2024
A video game experiment on mental time-travel and one-shot learning (taken)
Mental time-travel is the process of vividly remembering past personal experiences or imagining oneself in a future situation. Whereas humans can be asked to describe what they experience during mental time-travel, indirect approaches are needed to investigate whether mental time-travel exists in other species. We study a class of behavioural tasks that humans can presumably solve using mental time-travel, and that feature a behavioral readout other than verbal descriptions of subjective experiences. For example, a subject may perform an action to prepare for an event in the near future, by recalling a related but unique prior episode where they were unprepared.
In this project, we will design simple video game implementations of this behavioral paradigm, to study in rodent and human subjects. The project consists of four tasks. First, design and test implementations of the task using the Unity game engine. Second, run pilot human behavioural experiments with lab members and friends. Third, run the experiment online (e.g. on prolific.co) or with EPFL students. Four, analyse the behavioural data.
Good programming skills are a strict requirement; familiarity with Unity is an asset but not required. Interested students should send their application, including CV and grades, to both [email protected] and [email protected].
This project can also be done as a Semester project.