Semester Projects

Contact : Dr. Pearl Pu – [email protected]

Project duration : Spring 2025

Types : Bachelor thesis, master thesis, semester projects

Objective: Design an AI-powered travel agent within Microsoft Copilot that facilitates a “three-way conversation” between the user, the AI, and online travel data sources. This agent will take initial travel ideas from the user, retrieve and adjust options based on real-time data, and refine the itinerary collaboratively with the user. The goal is to create an engaging, efficient way for users to plan travel with minimal back-and-forth.

Unique Focus Areas:

• Conversational Flow Design: Create a natural, intuitive “conversation” experience where the AI actively interprets user objectives (destinations, budget, timeframe) and seamlessly retrieves options based on real-time travel data.
• Dynamic Data Integration: Design the AI to query and update itinerary elements from online sources (e.g., flights, hotels) in real-time, presenting users with accurate, up-to-date options for cost, availability, and timing.

Key Features:

1. User Objectives Intake: The agent initiates the conversation by gathering basic trip details, such as preferred destination, travel dates, and budget constraints.
2. Data Retrieval & Iterative Refinement: Once the AI understands user preferences, it queries online travel sources for options that align with these criteria, iterating and refining based on user feedback.
3. Collaborative Notifications: The agent only notifies the user when an itinerary choice requires input or when a key milestone (e.g., budget alignment, reservation details) is reached.

Deliverables:

• Video Prototype: Develop a video showcasing the three-way interaction in action, from initial travel concept through real-time data integration and iterative refinement.
• User Flow Diagram: Illustrate the agent’s conversational flow with both the user and data sources, detailing how preferences guide the itinerary curation.
• User Feedback Insights: Gather insights from user testing, focusing on the naturalness of the interaction, data relevance, and ease of use.

Evaluation Criteria:

• Does the agent reduce manual effort in itinerary planning by using online data intelligently?

Is the three-way conversational flow smooth, minimizing the need for excessive user inputs?

Contact : Dr. Pearl Pu – [email protected]

Project duration : Spring 2025

Types : Bachelor thesis, master thesis, semester projects

Objective: Design an AI-driven email agent that autonomously manages multi-step customer service interactions, resolving user issues across multiple exchanges. This agent should proactively follow up on queries, track case history, and keep users informed as needed.

Unique Focus Areas:

• State-of-the-Art Research & Innovation: Conduct research to understand existing AI-driven customer service solutions. Investigate how similar solutions operate, identify gaps, and propose innovative features or improvements that make this agent unique and effective for complex issue resolution.
• Long-Term Issue Management & Contextual Consistency: Emphasize the agent’s ability to track complex service issues over multiple interactions, maintaining context and delivering accurate responses based on historical data.

Key Features:

1. Autonomous Issue Resolution: Build workflows that allow the agent to autonomously resolve common service issues, including follow-ups and customer-specific updates.
2. Timely and Relevant Updates: Design the agent to notify users only when specific input or approval is required, reducing inbox clutter.

Deliverables:

• Innovation Research Report: Provide a research report that outlines existing solutions in autonomous customer service, comparing them to your design and identifying unique innovations.
• Video Prototype: Produce a video showcasing the agent managing a customer issue resolution scenario within Microsoft Copilot.

Evaluation Criteria:

• How effectively does the agent handle complex customer issues with minimal input?
• Are the updates clear and helpful for the user?

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Kalpani Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch)

Student Name:
Zied Mustapha

Abstract:
We present an innovative approach for aspect-based emotion analysis in distress-related conversations utilizing large language models (LLMs), specifically the GPT-3.5 model. The focus of this study is on the design and implementation of advanced prompt techniques, including zero-shot, few-shot, and chain-of-thought prompting, to generate structured responses for further analysis. The method was tested on two distinct datasets: CounselChat, a collection of therapy dialogues, and the Reddit Empathetic Dia- logues (RED) dataset, a collection of peer support dialogues from Reddit dataset. By leveraging the power of LLMs, this study automates the task of emotion identification and corresponding cause categorization, and the extraction of Motivational Interview- ing Treatment Integrity (MITI) categories from professional responses, bypassing the need for labor-intensive manual labeling. A detailed analysis was performed on the frequency of emotions, their causes, and the corresponding MITI categories, leading to interesting observations about the relationship between emotions, causes, and treatment responses. One possible application is the evaluation of responses in the RED dataset, for which GPT-3.5 can be instructed to use the relationship between emotions, causes and MITI response categories as a reference and assess the responses accordingly (refer to figure 1). The study also introduced a modular and robust code architecture to handle a large number of subsequent requests to the GPT-3.5 model. The results obtained provide compelling evidence for the potential of LLMs in understanding and annotating distress-related conversations, contributing to the development of more empathetic and effective conversational agents for addressing psychological distress.

Completed During:
Spring, 2023

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Kalpani Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch) and Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Joshua Swanson

Abstract:
This study investigates the role of fairness in the acceptance and use of two-sided recommender systems, using video sharing platform TikTok as a case study. Through an online survey of 429 participants, we collected data on various aspects of users’ perceptions and usage habits of TikTok, including system and service quality, perceived fairness, perceived usefulness, and perceived ease of use. Our findings show that system quality had a positive and statistically significant influence on perceived fairness, while service quality did not have a significant impact. Additionally perceived fairness had a statistically significant on perceived usefulness. However, the relationship between perceived fairness and attitude towards use was not statistically significant. While the fit indices for our models were not ideal, these results provide insights into the complexity of the relationships between fairness and other factors that influence user behavior in the adoption and use of two-sided recommender systems.

Completed During:
Fall, 2022

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Jingrong Chen

Abstract:
This work introduces the methods to detect and generate humorous conversation based on conversational data gathered from different resources. In order to gather and filter the humorous conversation data, we combined Incongruity-Based Features and a neural-based method to filter the data. To find humorous conversation data, our method is useful. In the later part of our paper, we implemented three main types of text generation models to compare with our finetuned GPT-J model; our finetuned model outperforms the traditional methods like LSTM and Seq2Seq models; It only requires a few datasets compared to methods, but it performs better. It can generate humorous replies according to the context question, just like a human.

Completed During:
Spring, 2022

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Supervisor:
Dr. Pearl Pu

Assistant:
Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Mauricio Byrd Victorica

Abstract:
Fairness in machine learning systems has become a highly active area in the past few years, as the ethical concerns surrounding automated decision-making become more prominent and these systems become more commonplace. Recommendation systems are not the exception: they operate in different contexts, from seemingly innocuous content recommendation to more sensitive areas such as targeted job offers. Although most of the literature on recom- mendation fairness focuses on whether users receive fair recommendations, in the context of platforms that mediate between said users (or consumers) and producers (or creators) whose livelihood is directly impacted by the recommendation system in place, it becomes crucial to consider two-sided fairness. This project focuses on the specific case study of the video- sharing application TikTok and the perceptions of fairness in recommendation from content creators and consumers in the platform. In particular, the aim is to explore insights from online user reviews, not only from TikTok but also from Youtube, another prominent video- sharing application and website; the idea is to gain a broader overview with this comparative study format.

Completed During:
Fall, 2021

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Curate a benchmark dataset for testing novel evaluation metrics of conversational chatbots

Assistant:
Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Siran Li

Keywords:
Dataset curation; conversational chatbots; natural language processing

Abstract:
Evaluation of conversational chatbots is an open research problem within the NLP community. Previous studies tested various automatic metrics for a proxy of human evaluation of chatbot’s naturalness. While several popular automatic metrics correlated poorly with human judgment (Liu et al., 2016), perplexity demonstrated promising results (Adiwardana et al., 2020). However, the notion of naturalness in the aforementioned study did not include a set of essential human-like conversation attributes, e.g., entertainment or empathy, as suggested by the PEACE model (Svikhnushina and Pu, 2021). The aim of this project is to create a benchmark dataset of conversations with sufficient coverage of the PEACE constructs that could be further used for evaluation and comparison of different conversational models as well as testing of novel evaluation metrics.

The student is expected to:

• Survey existing popular open-domain chatbots whose conversational responses could serve as a reasonable baseline.
• Create a benchmark dataset of conversations in a similar way as described in (Adiwardana et al., 2020).
• Obtain human judgments for different conversational aspects for the curated data via crowdsourcing.

Related Skills:
Knowledge in natural language processing, data mining, and machine learning; strong analytical skills; programming skills (knowledge of Python is essential, basic web development skills is a plus).

Suitable for:
Master student. Interested student should contact Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2021

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Application of natural language processing tools and methodologies to generate reflections and paraphrases of distress stories in Reddit

Assistant:
Kalpani Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch)

Student Name:
Zheng Wang

Keywords:
Entity-identification; Emotion recognition; Paraphrasing

Abstract:
Most people suffer from emotional distress due to going through a significant life change, financial crisis, being a caregiver or various physical and mental health conditions. However, due to public and personal “stigma” associated with mental health, most people do not reach out for help. Even therapeutic consultations are limited and are not available 24/7 to support people when they are going through a traumatic episode. Therefore, it is important to assess the ability of AI-driven chatbots to help people deal with emotional distress and help them regulate emotions.

In recent times, more and more research is focused on how to generate controlled chatbot responses rather than generic responses produced by end-to-end response generation models. In such an era, it is of interest to look at ways and means of how practices in counseling could be incorporated into the chatbot response generation process. One of the most important dimensions of counseling is “reflection and paraphrasing” (Kagan and Evans, 1995). It lets the speaker know what you have understood and communicates empathy. This is achieved by the listener by both repeating and feeding a shorter version of their story back to the client as well as reflecting on their emotions. An example of a distress story and a generated reflection and paraphrase would be as follows:

Distress story:
“My mother is getting sick. She is alone in her village and only has one of my brother’s children staying with her. But I’m not sure the boy is taking good care of her. I am so worried because they are far from the hospital and he will not know what to do if she gets sicker.”

Reflection and paraphrase:
“It sounds like you are incredibly anxious at the moment, worrying about your mother’s health. You also seem to be concerned that the boy staying with her will be unable to look after her if necessary.”

Even though the above dimension is quite vaguely analyzed in human-human conversations involving crisis counseling (Zhang and Danescu-Niculescu-Mizil, 2020) and empathetic conversation strategies (Welivita and Pu, 2020; Sharma et al., 2020; Pfeil and Zaphiris, 2007) (identified by different terms that refer to the same concept such as backward-orientation, acknowledgment, interpretation, understanding, etc.), generation of detailed reflections and paraphrases that include contextual information, which makes the listener more heard of, has not been investigated. This project would address this concern by analyzing a large-scale distress-related dialogue dataset curated from a carefully selected subset of subreddits and investigating natural language processing tools and methodologies that could help us generate appropriate, contextually relevant reflections and paraphrases out of them so that they could be incorporated into the chatbot response generation process.

The student is expected to:

• Apply natural language processing tools and methodologies to extract named entities and identify the characters involved in the story.
• Identify emotional reactions present in the story, towards whom they are directed and the causes behind.
• Generate reflections and paraphrases based on the above information.

References:

• Kagan, C. and Evans, J., 1995. Counselling. In Professional Interpersonal Skills for Nurses (pp. 129-148). Springer, Boston, MA.
• Zhang, J. and Danescu-Niculescu-Mizil, C., 2020. Balancing objectives in counseling conversations: Advancing forwards or looking backwards. arXiv preprint arXiv:2005.04245.
• Welivita, A. and Pu, P., 2020. A Taxonomy of Empathetic Response Intents in Human Social Conversations. Proceedings of the 28th International Conference on Computational Linguistics.
• Sharma, A., Miner, A.S., Atkins, D.C. and Althoff, T., 2020. A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
• Pfeil, U. and Zaphiris, P., 2007, April. Patterns of empathy in online communication. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 919-928).

Related Skills:
Knowledge in machine learning and natural language processing; Ability to code (preferably in Python); Experience in training and evaluating neural network models is preferable.

Suitable for:
Master student. Interested student should contact Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2021

Project report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Kalpani Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch)

Student Name:
Sena Necla Çetin

Abstract:
Recently, AI-driven chatbots have gained interest to help people deal with emotional distress and help them regulate emotions. However, since conversational data between patients who are in emotional distress and therapists who are actively offering emotional support is hardly available publicly due to privacy and ethical reasons, the most feasible option is to train chatbots on data from online forums such as Reddit. One challenge is ensuring that the data collected from these platforms contain responses that lead to high engagement and satisfaction and avoid those that lead to dissatisfaction and disengagement. We have developed a novel scoring function that can measure the level of satisfaction and engagement in distress-oriented conversations. Using this scoring function, we classified dialogues in the Reddit Emotional Distress (RED) dataset as highly satisfying, less satisfying, highly engaging, and less engaging. By analyzing these separate dialogues, we finally came up with a set of guidelines that describes which conversational strategies lead to highly satisfying and highly engaging conversations and which conversational strategies lead to less satisfying and less engaging conversations. Our guidelines can serve as a set of rules when developing therapeutic chatbots from online mental health community data so that inappropriate responses could be avoided and speaker satisfaction and engagement with these chatbots could be increased.

Completed During:
Spring, 2021

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Iuliana Voinea

Abstract:
This project proposes to preprocess the existing EmpatheticDialogues dataset, consisting of 25K conversations, into a new version that comprises dialogues which contain empathetic questions in the final listener’s turn. The dialogues were then annotated according to the empathetic question in the final turn, following a fine- grained taxonomy of 9 question types and 12 question intents. The annotation was performed through a set of techniques: manual annotation, pre-trained sentence- similarity classifiers based on siamese and triplet networks, and BERT-based classifiers. This project describes the workflow employed to preprocess the original empathetic dialogues, perform manual annotation on a small subset of dialogues, apply the pre-trained sentence-similarity classifiers to extend the manually-annotated data by a number of additional examples, and train the BERT-based classifiers to annotated the rest of the data points. Although the labels were highly unbalanced for both question types and intents in the extended manually-labelled dataset, the BERT-based classifiers achieved accuracies of 78% for types and 75% for intents, eventually being used to label the rest of the data points. The resulting dataset was confirmed to follow the typical human social interaction patterns after its quality evaluation was performed through visualisation techniques.

Completed During:
Spring, 2021

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Junze Li

Abstract:
Humor recognition has been widely studied as a text classification problem using data-driven approaches. However, most existing work does not examine the actual joke mechanism to un- derstand humor. We break down any joke into two distinct components: the set-up and the punchline, and further explore the special rela- tionship between them. Inspired by the incon- gruity theory of humor, we model the set-up as the part developing semantic uncertainty, and the punchline disrupting audience expectations. With increasingly powerful language models, we were able to feed the set-up along with the punchline into the GPT-2 language model, and calculate the uncertainty and surprisal values of the jokes. By conducting experiments on the SemEval 2021 Task 7 dataset, we found that these two features have better capabilities of telling jokes from non-jokes, compared with existing baselines. We also took part in the Se- mEval 2021 Task 7 (HaHackathon: Detecting and Rating Humor and Offense) by exploring and comparing different deep learning struc- tures to recognize humor and offensiveness. Our DeBERTa model ranks top 3 in every sub- task on the development phase leaderboard.

Completed During:
Fall, 202

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Curate and analyse a large-scale dialogue dataset containing emotional support for people in distress, which can potentially be used to train a mental care giving chatbot.

Assistant:
Kalpani Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch)

Student Name:
Chun-Hung Yeh

Keywords:
Emotional support; Web crawling; Natural language processing

Abstract:
Most people suffer from emotional distress due to going through a significant life change, financial crisis, being a caregiver or due to various physical and mental health conditions. Inability to regulate emotion in such episodes can potentially lead to self-destructive behavior such as substance abuse, self-harm or suicide. However, due to public and personal “stigma” associated with mental health, most people do not reach out for help. Even therapeutic consultations are limited and are not available 24/7 to support people when they are going through a traumatic episode. Therefore, it is important to assess the ability of AI driven chatbots to help people to deal with emotional distress and help them regulate emotion. One of the major limitations in developing such a chatbot is the unavailability of a curated dialogue dataset containing emotional support. With this project, we aim to curate and analyse such a dataset having the potential to train and evaluate mental care giving chatbot that can support people in emotional distress.

The student is expected to:

• Explore sources in the web from which we can collect large sets of conversations containing emotional support.
• Develop a web crawler to crawl potential sources in the web.
• Clean the conversations and develop a final dataset containing dialogues offering emotional support to people in distress.
• Perform analysis on the final dataset and come up with a preliminary taxonomy of response intents.

Related Skills:
Basic knowledge in natural language processing; Ability to code (preferably in Python); Experience in web crawling is preferable.

Suitable for:
Master student. Interested student should contact Anuradha Welivita (kalpani DOT welivita AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2020

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Identify the main drivers of people’s engagement with open-domain chatbots and explore various aspects of current interaction experience with them based on user reviews from Google Play market and Apple app store.

Assistant:
Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Alexandru Placinta

Keywords:
Qualitative research; social chatbots; motivations; remote experience evaluation

Abstract:
Due to recent advances in neural network-based language generation the area of open-domain chatbot development has become increasingly active. To ensure compelling user experience the design of conversational agents should meet eventual user goals and expectations. A number of studies in HCI community explored user motivations, needs, and perceptions of chatbots through a variety of methods, including surveying (Brandtzaeg, 2017), interviewing (Jain, 2018; Clark, 2019), diary studies (Muresan, 2019), and review analysis (Purington, 2017). However, previous works were either focusing on chatbots in general (also including task-oriented ones) or conducting a case study of a single specific agent. The aim of this project is to identify core user motivations and significant interaction experience aspects with open-domain chatbots based on elaborate analysis of reviews and ratings that users provided for a range of trending conversational apps on popular software distribution platforms.

The student is expected to:

• Survey existing popular open-domain chatbots.
• Scrape user reviews and ratings of surveyed chatbots from Google Play market, Apple app store, or other platforms.
• Analyse the reviews using both ML-based automatic techniques (e.g., filtering methods, sentiment analysis, topic analysis, etc.) and qualitative and/or quantitative methods (e.g., coding, affinity diagramming, thematic analysis; correlation analysis, statistical analysis, etc.) to elicit main user motivations, experiences, and concerns of open-domain chatbots.

Related Skills:
Knowledge in Data Mining and/or Machine Learning; strong analytical skills; familiarity with natural language processing; statistical analysis basis

Suitable for:
Master student. Interested student should contact Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2020

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Create a taxonomy of English interjections with exhaustive lists of examples that could serve for enhancing the naturalness of chatbot’s responses.

Assistant:
Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)

Student Name:
Jean-Baptiste De La Broïse

Keywords:
Interjections; discourse analysis; taxonomy; natural language processing

Abstract:
Interjections are words and expressions that people use to communicate sudden reactions, feelings, and emotions. Most of the time we use interjections unconsciously and perceive them as an integral part of a human conversation. As naturalness is an important aspect for open-domain conversational agents, active research is being conducted on the role of interjections for chatbots. Earlier studies demonstrated that chatbots are evaluated as more natural and engaging when they use interjections in their responses (Marge, 2010; Cohn, 2019). However, previous works operated with limited lists of interjections and used custom heuristic rules and classifications to introduce them into agent’s utterances. The aim of this project is to create a robust taxonomy of interjections enriched with numerous examples so that it could be used reliably to enhance naturalness of chatbot’s responses.

The student is expected to:

• Survey the Linguistics literature on the subject of interjections in human conversations, their form, position, and meanings.
• Come up with a fine-grained taxonomy of English interjections with exhaustive lists of examples for each class. Special attention should be paid to emotive interjections (e.g. positive surprise: Wow!, Whoa!; disgust: Yuck!, Ugh!; etc.).
• Validate the taxonomy using the corpus of emotional human dialogs.

Related Skills:
Strong analytical skills; background and/or interest in discourse analysis; familiarity with natural language processing

Suitable for:
Master student. Interested student should contact Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2020

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Zhechen Su

Abstract:
The rising demand for artificial intelligence-powered chatbots with sentiment analysis is creating new growth opportunities for numerous areas. Thus, building empathetic natural language processing agents becomes an interdisciplinary field of natural language processing. Some researchers presented the seq2seq model to make responses more emotional, while others tried the generative model for more variation. However, it is arduous to control the emotion and sentiment of generated sentences. In this paper, we focus on the need for the variational empathetic chatbot. The model combines the plain Transformer chatbot model and Conditional Variational Autoencoders (CVAE). With the help of neural emotional classifiers and pre-trained weights from RoBERTa, our model achieves the best score in automatic and human evaluation. Experimentally, we show in the quantitative and qualitative analyses that the proposed models can successfully generate high-quality abstractive conversation responses following designated emotions.

Completed During:
Fall, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Junze Li

Abstract:
Human-machine interaction, particularly via dialogue system, was a popular re- search area in the past decade. One of the challenges for dialogue systems is to recognize feelings and topics in the multi- turn conversation partner and reply ac- cordingly, which is a key communicative skill in human-human interaction. This project aims at recognizing and acknowl- edging the speakers’ feelings and topics in conversations, by leveraging the novel EmpatheticDialogues dataset proposed by Facebook AI Research. We mainly fo- cus on incorporating topic information into the Transformer framework to gen- erate more informative and interesting re- sponses. Firstly, we implement the Topic- prepend model, which inserts the topic information at the beginning of dialogue sentences based on the pre-trained topic classifier. Secondly, we implement the Topic-aware model which adopts the joint attention mechanism and biased genera- tion probability. The topic words are ob- tained by a pre-trained LDA topic model. Finally, we conduct experiments to com- pare our models with several baselines by both automatic and human evaluation. The results indicate that our models yield bet- ter performance by deepening and widen- ing the chatting topic.

Completed During:
Fall, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Build a dialog model that can produce character-specific response.

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Yueran Liang

Keywords:
human dialog modeling, natural language processing

Abstract:
One of the main characteristics of human dialogs is the diversity of the responses: given an input message, one could have many responses that are equally good. This could be explained by the fact that people with different backgrounds and personalities could respond differently to the same utterance. This semester project aims at modeling different personas in data-driven approaches.

The student is expected to:

• Reproduce the existing work and train the model on the provided Big Bang Theory corpus.
• Evaluate the model performance.

Related Skills:
Basic knowledge in neural networks and natural language processing

Suitable for:
Master student. Interested student should contact Yubo Xie (yubo DOT xie AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Build a neural model that can generate Chinese couplet response.

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Zou Xiaoyan

Keywords:
Chinese couplets, sequence transduction, natural language processing

Abstract:
Chinese couplets are a reduced form of Chinese poems, composed of two lines of poetry, adhering to some rules. Dueling couplets has been a popular game among intellectuals since ancient China, where one composes the second line according to the given first line. This can be regarded as a sequence transduction problem. This semester project aims at teaching machines to compose the second lines in couplets, using data-driven approaches such as neural networks.

The student is expected to:

• Find or create a Chinese couplet dataset.
• Use some sequence transduction architecture (for example seq2seq) to build a model.
• Evaluate the results.

Related Skills:
Proficiency in Chinese; basic knowledge in neural networks

Suitable for:
Master student. Interested student should contact Yubo Xie (yubo DOT xie AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Fall, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Oussama Abouzaid

Abstract:
When developing chatbots, the customer experience should always be the important thing. That is why it is important to ensure seamless collaboration between people and technology. In this project, we introduce a transactional chatbot that helps a user order beer, while keeping the conversation as human as possible, by introducing emotions and small talk.

Completed During:
Spring, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Ayyoub EL AMRANI

Abstract:
Human verbal communications and dialogues always hide several intentions and emotions.[1] Lots of research have been made in this field regarding the English language. However, the French language knows poor interest in this area. That’s why we were highly interested in implementing a french language model able to add emotions in generating sentences from a certain input of sentences. However, this ultimate goal was difficult to reach in the given time slot. Hence, in this report, we will be presenting the results we find regarding the work made from scratch. Indeed, we first had to look for french data on which the model can be based. We then proceed to work on parsing and cleaning the huge amounts of data found. We then did some basic statistical analysis of our data. We finally tried to test a ”sequence to sequence” -hoping this can be integrated into a chat-bot later- model on a small sample of our data to have an idea of the ”quality” of our data. In parallel with this work, we did some research in order to test the validity of some potentially ”good” emotion dictionaries. Indeed, we compared DeepMoji and LIWC dictionary and found out that LIWC dictionary was performing very poorly.

Completed During:
Spring, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Goals:
Devise an algorithm that can automatically segment dialogs in subtitles.

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Yan Fu

Keywords:
human dialogs, text segmentation, natural language processing

Abstract:
One of the main challenges of open-domain dialog modeling is the scarcity of training data, especially for multi-turn settings. Movie and TV subtitles are naturally a good source for developing conversation corpora. Currently the biggest corpus is the OpenSubtitles dataset. However, subtitle files usually lack clear scene markers, making it difficult to extract self-contained dialogs used for training multi-turn dialog models.

The student is expected to:

• Preprocess and analyze the OpenSubtitles dataset
• Devise an automatic dialog segmentation algorithm (rule-based or data-driven)
• Evaluate the segmentation accuracy

Related Skills:
Basic knowledge in natural language processing

Suitable for:
Undergraduate/Master student. Interested student should contact Yubo Xie (yubo DOT xie AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.

Completed During:
Spring, 2019

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Francis Ikpe Ogar Damachi

Abstract:
This work explains the process when implementing the baseline and affect language model. We are interested in training an affect language model that can generate sentences in a specific affect category. These sentences should be grammatically and semantically correct. Furthermore, we want to see if the affect language model achieves lower perplexity scores compared to the baseline model. For that reason, we train both models and compare their scores. When training the affect language model, we use the LIWC api to generate a vector in order to represent if a word is emotionally colorful or not. In fact, this api is crucial in training the affect LM.

Completed During:
Fall, 2018

Project Report:

Duration:
One Semester

Lab:
HCI/IC/EPFL

Assistant:
Yubo Xie (yubo DOT xie AT epfl DOT ch)

Student Name:
Quentin Bacuet

Abstract:
In this work, we trained a special type of Language model, the Affect-Language Model that takes into account the affect part of the sentences. This model will be used to perform sentence generation for a specific affect category (Anger, Positive Emotion,…). The model was trained on two different datasets with two different architectures using LSTMs. Finally, human testing was used to assert the validity of both of them.

Completed During:
Fall, 2018

Project Report:

Duration:
One Semester

Goals:
Using tools like D3 to build an environment to let users explore, interact, and understand what their food journey has been in the past year, and how they are correlated with their emotion and mood.

Responsible TA:
Onur Yuruten (onur DOT yuruten AT epfl.ch)

Student Name:
Yumeng Hou

Keywords:
information visualization, machine learning, sentiment analysis, personal health

Abstract:
Existing apps do not do justice to the amount of attention we pay to personal food and nutrition intake. We spend three times a day to select, prepare, and consume our meals. And if we are able to share with our friends and family members, these meals can contribute to the most memorable moments of our lives. How can we make a digital diary of our food consumption and the emotions that accompanied them? how can we make these memories as delicious as the food we ate? This project aims at exploring both information visualization issues as well as personal health issues. Along the way, we want our students to have fun with the D3 tool and learn a few things about color design.

The responsible student will:

• Survey what has been in this field including work from our lab
• Build the food and mood visualizaiton environment in D3
• Make it available to the Web users

Related Skills:
Human computer interaction, information visualization, interest in personal health, web programming, passion to succeed in multidisciplinary work

Suitable for:
Master Student. Please contact Onur (above) or [email protected] along with your CV.

Duration:
One Semester

Goals:
Create and evaluate an EEG-based seizure forecasting system.

Responsible TA:
Igor Kulev (igor DOT kulev AT epfl.ch)

Student Name:
Yuguang Yao (exchange student from Tsinghua University)

Keywords:
time series, classification, forecasting, healthcare

Abstract:
Epilepsy afflicts nearly 1% of the world’s population, and is characterized by the occurrence of spontaneous seizure. Seizure forecasting systems have the potential to help patients with epilepsy lead more normal lives. In order for EEG-based seizure forecasting systems to work effectively, computational algorithms must reliably identify periods of increased probability of seizure occurrence. If these seizure-permissive brain states can be identified, devices designed to warn patients of impeding seizures would be possible. Patients could avoid potentially dangerous activities like driving or swimming, and medications could be administered only when needed to prevent impending seizures, reducing overall side effects. In 2014 and 2016 Kaggle completed two seizure prediction challenges. The first challenge primarily involved long-term electrical brain activity recordings from dogs. The second challenge focuses on seizure prediction using long-term electrical brain activity recordings from humans obtained from the world-first clinical trial of the implantable NeuroVista Seizure Advisory System. In this project you will apply and compare different methods on the EEG data obtained from both Kaggle challenges.

The responsible student will:

• Survey time series classification techniques
• Apply existing techniques on the EEG data
• Analyze the performance of these techniques

Related Skills:
Knowledge in Data Mining and/or Machine Learning; Programming skills

Suitable for:
Master Student

Duration:
One Semester

Goals:
Analyze relations between nutrition and emotional well-being based on the experiences reported in social media.

Supervisor:
Dr. Pearl Pu

Student Name:
Othman Benchekroun

Keywords:
Emotion Recognition, Twitter, Data Analysis, Statistics, Social Media Analysis, Nutrition, Well-being

Abstract:
Nowadays, much about human behavior can be discovered from their online traces in social media. People describe their lives, personal events, their reactions to products and global events, and also reveal their eating preferences. Research studies suggest that having a healthier lifestyle, including eating healthier food, can improve personal well-being and make a person happier. This project will analyze the relations between nutrition and emotional well-being as detected from social media. It will aim to derive more fine-grained patterns of relations between the products we eat and the emotions we experience.

The responsible student will:

• Survey the state-of-the-art methods in text-based emotion recognition and related work on personal well-being relation with nutrition
• Collect the tweets for studying relations between reported emotions and mentioned food types
• Adapt (and potentially improve) existent tools for emotion recognition and food mention classification to extract required information from the collected tweets
• Perform statistical data analysis to discover patterns of relations between emotions and specific food types

Related Skills:
Programming skills; Statistics; Basic knowledge of Data Mining and/or Machine Learning; Interest in Social Media Analysis, and/or Computational Linguistics

Suitable for:
Master Student

Duration:
Two semesters (February – June 2015, February – June 2016)

Goals:
Investigate the effects of different linguistic modifiers on emotional expressions, and suggest how to model those effects within emotion recognition system.

Responsible TA:
Valentina Sintsova and Pearl Pu

Student Names:
Margarita Bolívar Jiménez (2015) and Nataniel Hofer (2016)

Keywords:
Emotion Recognition, Text Classification, Social Media Analysis, Modifiers Effects

Abstract:
People express their emotions and feelings in multiple subtle ways. Even when they use explicit emotional terms, such as “happy” or “sad”, the emotional meaning of statements can change because of the variety of linguistic modifiers. Those include negation, intensity shifting, modality, and others. So far the researchers have investigated the effects of those modifiers on polarity of terms (positive vs. negative). However, their effects on more fine-grained emotion categories remain understudied. The first part of this project investigates the effects of different modifiers on emotional meaning of the terms via data analysis techniques. The second part studies to what extent the better modeling of modifiers improves emotion classification quality.

Project Report:

Duration:
One semester (February – June 2015)

Goals:
Design a human computation task that would allow collecting affective knowledge of better quality, and develop the evaluation techniques to quantify the impact of the task design.

Responsible TA:
Valentina Sintsova and Pearl Pu

Student Name:
Séphora Madjiheurem

Keywords:
Human Computation, Emotion Recognition, Crowdsourcing, Amazon Mechanical Turk, Experiment Design, Quantitative Evaluation Techniques

Abstract:
Social media are filled with emotional content, which many researchers and companies seek to analyze. However, automatic methods for emotion recognition are far from the level of human ability to understand emotion language. Human computation techniques are seen as a way to help machines learn how to detect emotions. Online labor platforms such as Amazon Mechanical Turk allow to use individual humans to obtain answers to such judgment tasks as emotion detection in text. One strategy to obtain quality answers is to combine answers from different workers. Yet, in order to make use of the wisdom of the crowd, human answers must be comparable. This can be achieved by providing clear instructions and designing tutorials for the task. Moreover, if the human computation task is subject to systematic bias, using multiple workers is not enough to obtain quality answers. In this project, two experiments were conducted in an online labor platform. The first experiment aimed to evaluate the impact of tutorials on the quality of the answers provided by workers and on their engagement in the task. The second experiment was focused on comparing the workers’ output quality and engagement when using different incentives for motivating workers. The results show that tutorials with limited instruction do not necessarily lead to poorer performance. The results also demonstrate better quality work from workers under certain treatment conditions for motivation depending on the difficulty of the task.

Duration:
One semester (September – December 2014)

Supervisor:
Dr. Pearl Pu

Student Name:
Renato Kempter

Goals:
Design a novel way to recommend interesting places for visiting in the touristic city based on the feedback from social media.

Solution:
Web application for exploring trending themes of city locations and finding a set of places to visit for each theme. The locations are extracted from the geo-localized photos of Instagram; and their themes are detected by topics of the associated hashtags.

Keywords:
recommender systems, topic modeling, instagram, travel recommendation, system design

Abstract:
People often have a subject, a taste, a style or an interest that guides them to visit places they like. Nevertheless, traditional online travel guides are giving recommendations rather based on the categories of the places and their rankings. In this project, we suggest to use user-generated content to aggregate additional information about locations. Using geo-localized photos from Instagram and their associated hashtags, we develop a system that: a) Automatically clusters a set of instagrams with respect to existent city locations, b) Extracts the trending topics in the hashtags associated with the photos, and c) Generates sets of diverse locations that share the same topic. As topics are extracted directly from the user-generated content, they are more dynamic than predefined category- or tag-based descriptions of locations and reflect user interests and context of visiting those places. The example topics we were able to extract are “Christmas,Christmastree” and “design,vintage,deco”. Finally, we design a web application where users can browse the various topics and their corresponding locations in order to discover locations that suit their taste, style, or interest.

Resources:

• Screenshots

Duration:
One semester (September – December 2014)

Supervisor:
Dr. Pearl Pu

Student Name:
Cédric Rolland

Goals:
Design a novel way to recommend and discover wines, which would not require prior knowledge of wine.

Solution:
iPhone application for user-centric wine recommendation, exploration and shopping. It features the tailored quiz for taste learning in order to discover wines a user might like. It also includes knowledge quizzes for teaching users wine-related concepts and a reward system for motivating users buying more wines and continuing using the application.

Keywords:
recommender systems, interface design, gamification, e-commerce, mobile application design

Abstract:
Wine recommendation is a default feature of multiple websites and applications selling wine. It usually asks the user to input their favorite bottles or give ratings to different wines, and then makes a recommendation based upon this information. However, it can be tedious for users, especially for those with little knowledge about wine. We wanted to design a new way of selling and presenting wine to customers, the way that would not require prior knowledge about wine and would be engaging for users. This resulted in the application called La Caveauté. It recommends wine to users based on the simple quiz about their taste preferences, such as for coffee, breakfast, juices, etc. The user answers to the multiple-choice questions, and the app adds the related tags to the user profile. The algorithm then recommends a wine based on those tags. In addition to the taste quiz and recommending wine, the application provides the full user experience for wine exploration, learning oenology and selecting the most appropriate wine for the moment. The design went through multiple iterations based on the collected user feedback.

Duration:
One semester (February – June 2012)

Responsible TA:
Dr. Zerrin Yumak

Student Name:
Javier Martin De Valmaseda

Goals:
Develop a stable platform for mobile sensor data collection, labeling and post visualization method.

Solution: Android mobile application for activity labelling and accelerometer data collection.

Keywords: mobile sensors, pervasive healthcare, time series analysis, ubiquitous computing, data collection

Abstract:
Activity recognition of every day activities can be used for many different purposes: preventive medicine, promotion of health-enhancing physical activities and a developing a healthier lifestyle. It has a high industrial interest and a high impact on society. In the same time wearable sensors are becoming less and less intrusive and more accurate. As well, mobile devices are becoming increasingly sophisticated and smartphones with accelerometer and other sensors are widely popular nowadays. In our study data have been collected by 8 users while performing normal daily activities along 2-3 days. They used two different types of sensors: BodyMedia and Affectiva together with an Android phone. We relied on the SAX method to process the data. Afterwards, it can be analyzed by machine learning algorithms to detect certain activity types. SAX is the first method for the symbolic representation of time series that allows dimensionality reduction and indexing with a lower-bounding distancemeasure. This symbolic approach allows a time series of length n to be reduced to a string length w (w < n).

Resources:
Project Poster

Duration:
One semester (February – June 2012)

Responsible TA:
Yu Chen

Student Name:
Alfredo Cerezo Luna

Goals:
Visualize and express emotions in online social environments.

Solution:
Embed emotion in Facebook profile pictures.

Keywords:
Interface Design, Emotion Visualization

Abstract:
We started by defining a library of 9 kineticons according to music related emotions, such as: transcendence, joyful, wonder, tenderness, nostalgia, pacefulness, energy, sadness and tension. We then implemented the kineticons in GroupFun, a group music recommender system that suggests a common playlist for users. Our evaluation was based on a live user study. Aiming at proving that our kineticons are well understood we showed to 15 users our design together with 3 words description of emotion and asked them to vote for the words that best describe each of the 9 kineticons. Additionally, we prepared a video of 9 emotions, each corresponding to a song. All 15 users watched the video and rated (from 1 to 5) the appropriateness of how the kineticons describe the emotions evoked by the song.