We work on four major topics:
- Machine Learning Tools for Behavior Analysis – We strive to develop computer vision and machine learning tools for the analysis and quantification of animal & human behavior. Published work in this field includes DeepLabCut, a popular open-source software tool for pose estimation. We have also developed state-of-the-art algorithms such as BUCTD – https://github.com/amathislab/BUCTD and HOISDF – https://amathislab.github.io/HOISDF/ .
- AI4Science with language models – We work at the interface of Natural Language Processing (NLP) and behavioral analysis to create visual-language models for behavioral analysis (WildCLIP – https://github.com/amathislab/wildclip). Together with the M.W. Mathis lab we built on GPT to create tools for translating natural language into Python code for analyzing behavioral data (AmadeusGPT – https://github.com/AdaptiveMotorControlLab/AmadeusGPT)
- Modeling of Sensorimotor Learning and Control – We develop normative theories of neural systems to elucidate their principles. For instance, we built task-driven models of the proprioceptive system to test hypotheses on sensorimotor processing (DeepDraw and Task-driven Proprioception – https://github.com/amathislab/Task-driven-Proprioception). We also use physics-informed machine learning to integrate principles of biomechanics and neural dynamics to our models of the sensorimotor system.
- Brain-inspired motor skill learning – Watching any expert athlete, it is apparent that brains have mastered how to elegantly control the body. This is an astonishing feat, especially considering the inherent challenges of slow hardware and the sensory and motor latencies that impede control. Understanding how the brain achieves skilled behavior is one of the core questions of neuroscience that we tackle through modeling using Reinforcement Learning and Control Theory. Check out DMAP, Lattice, our winning code for the MyoChallenge at NeurIPS 2022 as well as 2023!
We have many projects for each research direction that can be made suitable for bachelor projects, lab immersions, and master’s projects. Also, feel free to propose projects in the scope of the topics above.
Furthermore, together with Prof. Mackenzie Mathis’ lab, we are actively developing DeepLabCut and other tools. We have multiple projects available ranging from developing new features (e.g. improved active learning, novel GUI features), to software engineering (profiling and unit tests). You can find code for most of our projects on GitHub: https://github.com/amathislab
Please reach out to [email protected] and/or [email protected] with your CV, transcript and letter of motivation (addressing what topics you would like to work on and why), if you are interested!