The digital therapist: Acoustic changes in speech of people with depression could be used to help monitor mental health

Imagine this scenario: You've been feeling persistently blue lately, so you pull out your phone. Instead of asking Siri to tell you a joke, though, you open an app that records you simply talking about your day. A few hours later, your therapist sends you a message asking if you'd like to meet.

A program like this one that analyzes your speech and uses it to gain information about your mental health could soon be feasible, thanks in part to research from the University of Maryland showing that certain vocal features change as patients' feelings of depression worsen. The team behind the work will present their findings at the 168th meeting of the Acoustical Society of America (ASA), held October 27-31 at the Indianapolis Marriott Downtown Hotel.

The research is part of an interdisciplinary initiative at the University of Maryland to engineer patient-focused mental health monitoring systems. Rather than relying solely on patients' self-reports, these systems could monitor both physical and psychological symptoms of mental illness on a regular basis and provide both patients and their mental health providers with feedback about their status.

To conduct a quantitative experiment on the vocal characteristics of depression acoustician Carol Espy-Wilson and her colleagues repurposed a dataset collected from a 2007 study from an unaffiliated lab also investigating the relationship between depression and speech patterns. The earlier study assessed patients' depression levels each week using the Hamilton Depression Scale (a standard clinical evaluation tool to measure the severity of depression) and then recorded them speaking freely about their day.

The University of Maryland researchers used data from six patients who, over the six-week course of the previous study, had registered as depressed some weeks and not depressed other weeks. They compared these patients' Hamilton scores with their speech patterns each week, and found a correlation between depression and certain acoustic properties.

When patients' feelings of depression were worst, their speech tended to be breathier and slower. The team also found increases in jitter and shimmer, two measures of acoustic disturbance that measure the frequency and amplitude variation of the sound, respectively. Speech high in jitter and shimmer tends to sound hoarse or rough.

The researchers plan to repeat the study in a larger population, this time comparing speech patterns in individuals with no history of mental illness to those with depression to create an acoustic profile of depression-typical speech. A phone app could use this information to analyze patients' speech, identify acoustic signatures of depression and provide feedback and support.

Espy-Wilson hopes the interactive technology will appeal to teens and young adults, a particularly vulnerable group for mental health problems. "Their emotions are all over the place during this time, and that's when they're really at risk for depression. We have to reach out and figure out a way to help kids in that stage," she said.

Sometimes, patients might not recognize or be willing to admit that they are depressed. By receiving regular feedback based on acoustical and other measurements, they might learn to self-monitor their mental states and recognize when they should seek help. The technology could also promote communication between therapists and patients, allowing for continuous, responsive care in addition to regular in-person appointments.

The researchers acknowledge that developing an effective app requires a larger scope than just the underlying science—a challenge they plan to address. "We definitely need human factors to develop something that people will use," said Espy-Wilson. "There's a lot that has to go into making this a useful tool."

Note: Material may have been edited for length and content. For further information, please contact the cited source.

Acoustical Society of America (ASA)   press release