Russian Scientists Teach AI to Recognize Human Emotions More Accurately

Scientists from the Sberbank Center for Practical Artificial Intelligence and the Higher School of Economics have created a unique emotion recognition system that analyzes facial expressions, voice, and speech simultaneously. The technology combines temporal convolutional networks (TCN) and transformer multimodal architectures, which allows for more accurate tracking of changes in emotions over time. The system shows an increase in accuracy of more than 10% compared to models that work only with facial images. The development is applicable in business, security, and the social sphere.

The Center's scientific director Andrey Savchenko emphasizes that the system makes human-AI interaction more natural. The technology recognizes not only basic emotions, but also their dynamics in real conditions - in poor lighting, noise and other interference.

"We have learned not only to recognize basic emotions, but also to take into account their dynamics in real conditions - with different lighting, background noise and other interference. This is especially important for those business sectors where understanding customer emotions helps improve the quality of service. Our technology is already showing excellent results in tests, and in the future it can be adapted for virtual assistants, security systems and even telemedicine. The main advantage is flexibility. The audiovisual model works even with insufficient data, for example, when a face is not visible or a voice is poorly heard."

Andrey Savchenko, Scientific Director of the Center for Practical Artificial Intelligence of Sberbank

The technology is useful for call centers, marketing research, security systems, education and medicine. Scientists continue to improve the model to make emotion recognition even more accurate and adaptive.