1 Detección y regulación inteligente de emociones en ancianosJosé Miguel Latorre Postigo Unidad de Psicología Cognitiva Aplicada Instituto de Investigación en Discapacidades Neurológicas (IDINE) Universidad de Castilla-La Mancha
2 1. Esquema general del proyecto¿Qué objetivos pretendemos alcanzar?
3 ¿Qué pretendemos alcanzar?PRIMERO Desarrollar un sistema capaz de detectar emociones a través de la expresión facial y la respuesta fisiológica. Adaptar el sistema para su uso con ancianos. SEGUNDO Desarrollar un sistema de regulación de las emociones, mediante el uso de estímulos como color, iluminación, paisaje sonoros, música o recuerdos autobiográficos, entre otros.
4
5 EMOTION REGULATION Emotion Interpretation and RegulationRecognized EMOTIONS Ground truth data EMOTION REGULATION Data mining techniques for emotion regulation. Emotions & brain electrical and spatial activity data fusion. Data Fusion and Mining Emotion Interpretation and Regulation Emotion-tailored music / sounds, colors and lightning conditions.
6 AMBIENT ADAPTATION Smart Ambient Assessment Ambient IntelligenceEMOTION-TAILORED music/sounds, colors and lights; ROBOT BEHAVIORS AMBIENT ADAPTATION Ambient Intelligence An intelligent system is in charge of adapting the ambience towards regulating emotions. No private information is sent outside the system. Smart Ambient Assessment System tuned to regulate specific emotions in the ambience.
7 2. DETECCIÓN DE EMOCIONESProcedimientos para la inducción y detección de emociones
8 Emoción Conducta expresiva: comportamiento Activación fisiológicaExperiencia subjetiva: sentimientos Activación fisiológica Conducta expresiva: comportamiento
9
10
11
12 Detección de emociones a través de la expresión facialReconocimiento de emociones a través de la expresión facial en tiempo real Análisis de la Expresión Facial Detección de emociones a través de la Expresión Facial
13 Facial Expression Analysis1. Detection of facial points Active shape models are used for the detection of facial points ASM are statistics models about the possible shapes of an object Several models Databases: IMM: 37 images, 58 points BioID: 1521 images, 20 points XM2VTS: 2360 images, 68 points XM2VTS generates the most suitable model 1. Detection of facial points. ASM (Active Shape Models) are used for the detection of facial points. ASM are statistics models about the possible shapes of an object. Several models were tested for this purpose. Databases: IMM (Informatics and Mathematical Modeling): 37 images, 58 points BioID: 1521 images, 20 points XM2VTS (Extended Multi Modal Verification for Teleservices and Security): 2360 images, 68 points XM2VTS generates the most suitable model
14 Facial Expression AnalysisEyebrow Features Mouth-Nose Features Eye Features Here are some examples of geometrical features used by the ASM model for Feature Extraction. These consist mainly in calculating distances and/or angles between face parts. We have used: Eyebrow features Mouth-Nose Features Eye Features 2. Feature extraction
15 Data and Results: Emotion Detection in ImagesHits Fear Disgust Surprise Here are some examples of correctly classified emotions from facial expression. This is a hit for Neutral emotion. And here are the six basic emotions. Neutral Happiness Sadness Anger 4. Detection of emotions
16 Data and Results: Emotion Detection in VideoWrong Predictions Emotion: Happiness Predicted: Disgust Emotion: Anger Predicted: Disgust Emotion: Disgust Predicted: Anger This slide is intended to show that not all our predictions are correct. The main reasons that probably explain the prediction errors are: the ASM adjustment is incorrect, the pretended emotion is clearly not representative of the expected emotion the features between two emotions are very similar the transition from one emotion to another causes troubles during a short interval of time Emotion: Fear Predicted: Sadness Emotion: Surprise Predicted: Fear Emotion: Sadness Predicted: Fear
17 Detección de emociones a través de la respuesta fisiológicaRespuesta electrodermal Pulso Temperatura corporal Electromiograma (EMG)
18
19 Diseño de hardware Prototype Hardware scheme- I would like to remark that thjis is a working in progress, so despite all the hardware has been developed and it has been integrated, we are testing the whole system. Having this consideration, we will focus exclusively in the EDA sensor because time reason and because we are not put toguether al the phisiological signals.
20 Sensor de la actividad electro-dermalBasis: Sweat increases as arousal level grows. Constant voltage circuit measure skin conductance. 1.5 Hz low-pass filter and 0.05 Hz high-pass filter were used These signals were sampled at 20 Hz. This sensor is based in the fact that sweat increases as the arousal level increases. According with this, we located two disc on medial phalanges of left index and middle fingers, since the density of the sweat areas are higher in these locations
21 Technician monitor Patient monitor
22 Procesamiento de la señalOnce the EDA signals were recorded, they were post-processed by following the methodology described in [2]. Signals were filter again with a 1.5 Low-pass filter to decrease the noise. [2] Wearable and Automotive Systems for Affect Recognition from Physiology, Jennifer A. Healey. Phd dissertation
23 Procesamiento de la señal2) First forward difference was computed 3) Positive events were discriminated as those exceeding a threshold.
24 Detección de emociones en ancianosExperimentos con jóvenes/ancianos Cambios en la expresión facial con la edad Cambios en la respuesta fisiológica con la edad
25 3. Regulación DE EMOCIONESProcedimientos para el cambio del estado emocional a través de distintos estímulos ambientales
26 Description of the ExperimentThe experimentation is carried out in a specially organized room with white colored walls, where each participant is placed in front of a computer. Evaluation of color preference with the aim of regulating emotions is performed with a software application. The graphical user interface of the test is shown in Fig. 3. The test lasts up to 30 minutes, depending on the test participant who may answer the complete questionnaire quicker or slower. Each participant is asked to judge about each of the 32 images which are included in the image set. The set of images consists, first, of uniform or textured one-color pictures, and, second, of images taken form nature, mostly landscapes, where one dominating color is prevailing. The colors used in the pilot study are gray, light blue, pink, dark blue, light brown, brown, violet, green, light green, red, orange, and yellow. A sample of the image set is provided in this Figure.
27 First Musical Test: The BeatThe piece is titled “Walking on the Street”, framed in a suite called “Three Little Bar Songs Suite” The different melodies combine both classical and contemporary elements of music The only requirement is that both music pieces share a tonal harmonic language, with a harmonic rhythm of classical music and repetitive rhythmic parameters This enables to highlight each of the auditions to categorize them correctly So, in this way, we have a piece which rhythm uses constantly alternating dotted notes (providing a touch of swing) and syncopated notes in prominent places Then, changes are provided to the harmonic rhythm used.
28 Uso de técnicas de recuerdo autobiográficoDespués de realizar una Revisión de Vida estructurada se implementaría un sistema integrado de recuerdos personales, incluyendo: Fotografías Vídeos Música Paisajes sonoros …
29 4. Interacción sistema-usuario¿Cómo percibe e interactúa el usuario con tecnología desarrollada?
30 Participantes: Antonio Fernández-Caballero José M. LatorreMaría T. López Elena Lozano-Monasor Francisco Vigo-Bustos Marina V. Sokolova Alicia Fernández-Sotos Pablo Olivos Jara Arturo Martínez-Rodrigo Roberto Zangróniz José Manuel Pastor César Sánchez Melendez Laura Ros Juan Pedro Serrano Jorge Ricarte Luz Fernández-Aguilar
31 MUCHAS GRACIAS POR SU ATENCIÓN This work was partially supported by Spanish Ministerio de Economía y Competitividad / FEDER under TIN C2-1-R grant.