Affective Computing

Objective

To enhance the interpretation of human emotions through computer vision techniques.

Team

This is a collaborative work between:

• ADAPT SFI Research Centre
• University College Dublin

Description

Understanding human emotions is crucial for effective communication and interaction. This research leverages computer vision techniques to interpret emotions, focusing on facial expressions as primary indicators. By integrating insights from psychology and technology, the study explores how artificial intelligence can decode the complex interplay of physiological and psychological states reflected in facial cues.

Significant emphasis is placed on enhancing image captioning models by embedding emotional attributes from facial expressions. Traditional models often miss these emotional nuances, leading to less impactful captions. By integrating facial emotion recognition, this research aims to generate more expressive and authentic captions, providing a holistic representation of visual scenes.

We demonstrate the proposed framework for incorporating emotions into the image captioning process. [Das et al. 2023]

Additionally, the study advances student engagement recognition in educational settings. It proposes frame-level detection methods and knowledge transfer techniques to improve classification performance, addressing challenges posed by limited labeled data.

A comprehensive evaluation framework, incorporating subjective user feedback, assesses the quality of emotional captions. This study also explores innovative approaches for emotional color transfer and fusion caption generation, aiming to create more empathetic and contextually relevant interactions in artificial systems.

This research exemplifies the synergy between human emotions and technological advancements. By harnessing computer vision and AI, it aims to enrich human-machine interactions with empathy and understanding.

Results

Please refer to the publications.