Contact: DĂŒmbgen, Frederike ; Hoffet, Adrien
Synopsis: Implement a new way of interacting with your computer via voice control instead of the mouse and keyboard.
Level:BS, MS
Description: Google Home and Amazon Alexa are quickly revolutionizing how we interact with smart devices. Both use âwake wordsâ (âOK Googleâ and âAlexaâ respectively) to detect the userâs intention to interact. While the wake word detection is typically done on the device to insure minimum latency, the userâs commands following it are usually processed remotely.
The goal of this project is to program a microcontroller to process acoustic data locally and in real time. The microcontroller should run a speech recognition model to extract specific commands from the spoken words of the user. The chip should then emulate a USB device such as a mouse or keyboard buttons and send the derived commands to trigger actions on the host computer. An important aspect of the project will be to understand the limits of what can be processed on the microcontroller, in terms of memory and computation time.
The student has the option to either work on implementing machine learning models such as CNNs on the microchip, or to work on emulating the USB peripheral. Ideally, we will have two students working on both components of the project such that we have a full working system in the end of the semester.
Deliverables: A report and a working system with clear documentation.
References: for useful links, see list of URLs below.
Prerequisites: First part: Knowledge of or strong interest for machine learning, in particular neural networks. Basics in programming of embedded systems.
Second part: Basics of C programming, embedded systems, preferably knowledge of USB devices.
Type of Work: 50% algorithm design/analysis, 50% programming