Notre Dame Develops AI to Interpret Nonverbal Communication

In a press release, researchers at the University of Notre Dame detailed Voice4PIMD, a personalized AI system designed to interpret communication from people with profound intellectual and multiple disabilities.

Voice4PIMD uses cameras and sensors to capture facial expressions, eye gaze, body movements, heart rate, vocalizations and other signals. Caregivers train the system to associate those cues with meanings for each person. The researchers plan for it to provide text or audio notifications because the same expression, movement or sound can have different meanings for different people.

The project remains in its early research stages. The team is beginning additional testing with families through LOGAN Community Resources in South Bend, Indiana, and is also exploring uses for people with autism and dementia.