Validating ‘the theory of constructed emotion’ through AI simulations that show how emotions are formed.
A multimodal AI called ‘ML-MLDA’ (Multilayered Multimodal Latent Dirichlet Allocation) can now mimic the concept of emotion formation through simulations.
Researchers from the Nara Institute of Science and Technology (NAIST) and Osaka University have developed an innovative next-generation AI model that utilizes language (words), visual stimuli (exteroceptive), and physiological information (interoceptive) to construct emotions. (1✔ ✔Trusted Source
Study of Emotion Concept Formation by Integrating Vision, Physiology, and Word Information Using Multilayered Multimodal Latent Dirichlet Allocation
Go to source
)
This computational framework offers a potential approach to understand human psychology and emotion context.
The generative model demonstrates how humans organize past experiences and information in the mind to construct emotional concepts. In the future, the proposed ML-MLDA model could provide significant support in mental health care monitoring for patients with brain disorders such as dementia.
TOP INSIGHT
Did You Know?
By linking body signals, language, and visual information, #AI can now finally decode #human_psychology. The next-generation approach addresses a deep-rooted question: How are #emotions formed? #emotionAI #machinelearning #mentalhealth #psychology
Multimodal AI Processes Internal Bodily Signals and External Stimuli to Mimic Emotion
This model is based on the theory of constructed emotion, which proposes that emotions are not innate reactions but are built in the moment by the brain.
Emotions arise from integrating internal bodily signals (interoception, like heart rate) with external sensory information (exteroception, like sight and sound), allowing the brain to create a concept, not just a reflex.
“Although there are theoretical frameworks addressing how emotions emerge as concepts through information processing, the computational processes underlying this formation remain underexplored,” says Dr. Hieida.
To model this process, the research team used multilayered multimodal latent Dirichlet allocation (mMLDA), a probabilistic generative model designed to discover hidden statistical patterns and categories by analyzing how different types of data co-occur, without being pre-programmed with emotional labels.
Generative AI Model Can Independently Categorize Human Emotional States
The developed model was trained using unlabeled data collected from human participants who viewed emotion-evoking images and videos. The system was not informed about which data corresponded to emotions such as fear, joy, or sadness. Instead, it was allowed to identify patterns on its own.
29 participants viewed 60 images from the International Affective Picture System, which is widely used in psychological research.
While viewing the images, researchers recorded physiological responses such as heart rate using wearable sensors and collected verbal descriptions. Together, these data captured how people interpret emotions: what they see, how their bodies respond, and how they describe experiencing them.
When the trained model’s emotion concepts were compared with participants’ self-reported emotional evaluations, the agreement rate was about 75%. This was significantly higher than would be expected by chance, suggesting that the model categorized emotion concepts that closely matched how people experience emotions.
AI Could Assist in Monitoring Mental Health States in Patients with Dementia
By modeling emotion formation in a way that mirrors human experience, this research paves the way for more nuanced and responsive AI systems.
“Integrating visual, linguistic, and physiological information into interactive robots and emotion-aware AI systems could enable more human-like emotion understanding and context-sensitive responses,” says Dr. Hieida.
Moreover, because the model can infer emotional states that people may struggle to express in words, it could be particularly useful in mental health support, healthcare monitoring, and assistive technologies for conditions such as developmental disorders or dementia.
“This research has important implications for both society and industry, as it provides a computational framework that connects emotion theory with empirical validation, addressing the long-standing question of how emotions are formed,” concludes Dr. Hieida.
Reference:
- Study of Emotion Concept Formation by Integrating Vision, Physiology, and Word Information Using Multilayered Multimodal Latent Dirichlet Allocation – (https://ieeexplore.ieee.org/document/11071374)
Source-Eurekalert