Sistema software de análisis forense basado en el estudio emocional de la voz

Gabriel Elías Chanchí Golondrino; Liset Sulay Rodríguez-Baca; Luz Marina Sierra Martínez

doi:10.14482/inde.42.01.519.019

Authors

Gabriel Elías Chanchí Golondrino Universidad de Cartagena https://orcid.org/0000-0002-0257-1988
Liset Sulay Rodríguez-Baca Universidad Autónoma del Perú https://orcid.org/0000-0003-1850-615X
Luz Marina Sierra Martínez Universidad del Cauca https://orcid.org/0000-0003-3847-3324

DOI:

https://doi.org/10.14482/inde.42.01.519.019

Keywords:

affective computing, arousal, emotion models, forensic, interviews, valence

Abstract

Affective computing aims to create systems capable of recognizing; processing; and simulating human emotions to enhance human-computer interaction. Speech emotion recognition (SER) is a highly effective and non-invasive technique for assessing a user's emotions by analyzing physiological variables. However; despite its widespread use in end-user perception identification; few applications have been developed in the field of forensic analysis. To address this gap; this research proposes a new forensic emotion analysis software system; FOREMAN; based on the emotional study of the voice. The system was developed using the Iterative Research Pattern. FOREMAN determines the fluctuation of emotions in an audio file; as well as the percentage and spatial distribution of emotions present in its segments; using clustering methods. The system's effectiveness is demonstrated by its application to an audio file taken from a Colombian Special Jurisdiction for Peace (JEP) hearing.

Author Biography

Gabriel Elías Chanchí Golondrino, Universidad de Cartagena

References

S. Baldasarri, “Computación Afectiva: tecnología y emociones para mejorar la experiencia de usuario,” Revista Institucional de la Facultad de Informática, nº. 3, pp. 14-15, 2016.

G. E. Chanchí, M. Sánchez, and W. Y. Campo, “Sistema software para el análisis del estrés mental en test de usuarios,” Campus Virtuales, vol. 7, nº. 2, pp. 105-114, 2018.

D. M. Delgado, D. F. Girón Timaná, G. E. Chanchí, and K. Márceles Villalba, “Estimación del atributo satisfacción en test de usuarios a partir del análisis de la expresión facial,” Revista Ingenierías Universidad de Medellín, vol. 19, nº. 36, pp. 13-28, Jun. 2019, doi: 10.22395/rium.v19n36a1.

M. Plaza Torres, “Uso de ambientes virtuales y selección de parámetros de medidas en la aplicación para el tratamiento de fobias,” Ingeniería y Desarrollo, nº. 23, 2008.

Z. Ying, L. Xuefeng, G. Yu, and Y. Yifei, “Multi-Classifier Interactive Learning for Ambiguous Speech Emotion Recognition,” IEEE/ACM Trans Audio Speech Lang Process, vol. 30, pp. 695-705, 2022. doi: 10.1109/TASLP.2022.3145287.

B. Eduards and S. Askars, “Emotion Recognition in Real-World Support Call Center Data for Latvian Language,” in Salimbajevs, A. O. Smith-Renner A., Ed., CEUR-WS, 2022, pp. 200-223.

R. Ashrafidoost, S. Setayeshi, and A. Sharifi, “Recognizing Emotional State Changes Using Speech Processing,” in 2016 European Modelling Symposium (EMS), 2016, pp. 41-46. doi: https://doi.ieeecomputersociety.org/10.1109/EMS.2016.017.

R. Jahangir, Y. W. Teh, F. Hanif, and G. Mujtaba, “Deep learning approaches for speech emotion recognition: state of the art and research challenges,” Multimedia Tools and Applications volume, vol. 80, pp. 23745-23812, 2021. doi: https://doi.org/10.1007/s11042-020-09874-7.

G. Guruprasad, S. Poojary, S. Banu, and A. Alam, “EMOTION RECOGNITION FROM AUDIO USING LIBROSA AND MLP CLASSIFIER,” International Research Journal of Engineering and Technology (IRJET), vol. 08, nº. 07, pp. 21814-2817, 2021.

P. A. Babu, V. S. Nagaraju, and R. R. Vallabhuni, “Speech Emotion Recognition System With Librosa,” in 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT), IEEE, 2021, pp. 421-424. doi: 10.1109/CSNT51715.2021.9509714.

M. Neumann and N. Thang Vu, “Improving Speech Emotion Recognition with Unsupervised Representation Learning on Unlabeled Speech,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019. doi: 10.1109/ICASSP.2019.8682541.

G. Ramet, P. N. Garner, M. Baeriswyl, and A. Lazaridis, “Context-Aware Attention Mechanism for Speech Emotion Recognition,” in 2018 IEEE Spoken Language Technology Workshop (SLT), IEEE, 2018. doi: 10.1109/SLT.2018.8639633.

J. Posner, J. A. Russell, and B. S. Peterson, “The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology,” Dev Psychopathol, vol. 17, nº. 3, pp. 715-734, Jul. 2005. doi: 10.1017/S0954579405050340.

G. Mishne, D. Carmel, R. Hoory, A. Roytman, and A. Soffer, “Automatic analysis of call-center conversations,” in CIKM’05:Proceedings of the 14th ACM International Conference on Information and Knowledge Management, 2005, pp. 453-459. doi: https://doi.org/10.1145/1099554.1099684.

L. F. Parra-Gallego and J. R. Orozco-Arroyave, “Classification of emotions and evaluation of customer satisfaction from speech in real world acoustic environments,” Digital Signal Processing: A Review Journal, vol. 120, 2022. doi: 10.1016/j.dsp.2021.103286.

M. Lynn, C. Su, and K. Maw, “Recognition and Analysis of Emotion Types from Myanmar Movies,” in ICIIBMS 2018, Track 1: Image Processing, Computer Science and Information technology, Bangkok, Thailand, IEEE Computer Society Digital Library, 2018, pp. 306-3013. doi: 10.1109/ICIIBMS.2018.8550024.

B. Milana, D. Vlado, and K. Alexey, “Influence of Emotion Distribution and Classification on a Call Processing for an Emergency Call Center,” Telfor Journal, vol. 13, nº. 12, pp. 75-80, 2021. doi: 10.5937/TELFOR2102075B.

T. Deschamps-Berger, “Emotion Recognition In Emergency Call Centers: The challenge of real-life emotions,” in 2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), IEEE, 2021, pp. 1-5. doi: 10.1109/ACIIW52867.2021.9666308.

M. Bancroft, R. Lotfian, J. Hansen, and C. Busso, “Exploring the Intersection Between Speaker Verification and Emotion Recognition,” in 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2019, pp. 337–342. doi: 10.1109/ACIIW.2019.8925044.

G. Chanchí and A. Córdoba, “Análisis de emociones y sentimientos sobre el discurso de firma del acuerdo de paz en Colombia,” Revista Ibérica de Sistemas e Tecnologias de Informação, vol. E22, 2019.

F. Eyben, M. Wöllmer, and B. Schuller, “OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit,” in 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, Netherlands: IEEE Xplore, 2009, pp. 1-6. doi: 10.1109/ACII.2009.5349350.

G. E. Chanchí Golondrino, L. M. Sierra Martínez, and M. A. Ospina Alarcón, “Application of Affective Computing in the Analysis of Emotions of Educational Content for the Prevention of COVID-19,” International Journal on Engineering Applications (IREA), vol. 10, nº. 3, p. 209, May 2022. doi: 10.15866/irea.v10i3.21056.

R. Segundo and C. Ongil, “MPATIA-CM: Comprehensive Protection of Gender-based Violence Victims through Multimodal Affective Computingle,” Sociologyst and TechnoScience, vol. 12, no. 1, pp. 1-33, 2022. doi: 10.24197/st.1.2022.1-33.

J. Cicres, “Comparación forense de voces mediante el análisis multidimensional de las pausas llenas,” Signos, vol. 47, nº. 86, pp. 365-384, 2014.

Y. Karni-Visel, I. Hershkowitz, M. E. Lamb, and U. Blasbalg, “Emotional valence and the types of information provided by children in forensic interviews,” Child Abuse Negl, vol. 129, 2022. doi: https://doi.org/10.1016/j.chiabu.2022.105639.

L. A. Solarte Moncayo, M. Sánchez Barragán, G. E. Chanchí Golondrino, D. F. Duran Dorado, and J. L. Arciniegas Herrera, “Dataset de contenidos musicales de video, basado en emociones,” Ingenierías USBMed, 2016. doi: 10.21500/20275846.2460.

K. Pratt, “Design Patterns for Research Methods: Iterative Field Research,” 2009.

G. E. Chanchí, M. A. Ospina, and L. M. Sierra, “Application of Affective Computing in the Analysis of Advertising Jingles in the Political Context,” International Journal of Advanced Computer Science and Applications, vol. 13, nº. 4, 2022. doi: 10.14569/IJACSA.2022.0130465.

G. E. Chanchí-Golondrino, M. A. Ospina-Alarcón, and M. Monroy-Rios, “Analysis of advertising jingles from the ‘80s and ‘90s through affective computing,” Cuadernos de Administración, vol. 38, nº. 73, pp. 1-16, 2022. doi: https://doi.org/10.25100/cdea.v38i73.11153.