NYC Gazette

Since the COVID-19 pandemic began, videoconferencing has become an integral part of both professional and personal interactions. Platforms like Zoom, MS Teams, FaceTime, Slack, and Discord are used widely for work and socializing. As these platforms continue to play a crucial role in communication, understanding what makes virtual meetings enjoyable or efficient is becoming increasingly important.

Researchers at New York University have developed an artificial intelligence model designed to analyze human behavior during videoconferences. This model can predict whether meetings are perceived as fluid and enjoyable by examining conversational dynamics such as turn-taking and facial expressions.

"Our machine learning model reveals the intricate dynamics of high-level social interaction by decoding subtle patterns within basic audio and video signals from videoconferences," said Andrew Chang, a postdoctoral fellow in NYU’s Department of Psychology and lead author of the study. The research was presented at the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).

The development of this AI involved training it with over 100 person-hours of Zoom recordings. The model analyzed voice, facial expressions, and body movements to detect moments when conversations became less fluid or enjoyable. It found that "awkward silences" were more disruptive than instances where participants spoke over one another.

An independent team of more than 300 human judges also evaluated the same footage to validate the model's accuracy. Their assessments closely aligned with those made by the AI.

Dustin Freeman, a visiting scholar in NYU’s Department of Psychology and senior author of the paper, emphasized the significance of this research: "Videoconferencing is now a prominent feature in our lives, so understanding and addressing its negative moments is vital for not only fostering better interpersonal communication and connection but also for improving meeting efficiency and employee job satisfaction."

The study also involved contributions from Viswadruth Akkaraju and Ray McFadden Cogliano from NYU’s Tandon School of Engineering, David Poeppel from NYU’s Department of Psychology and Max Planck Society in Munich, Germany. Funding was provided by grants from various sources including the NYU Discovery Research Fund for Human Health.