Making a game of speech therapy for kids with cerebral palsy

Imagine outsinging the Phantom of the Opera and then freeing your friends by distracting a pack of trolls.

No, this isn’t the plot of a new fantasy video game; instead, it’s an innovative method to deliver speech therapy.

The game, called Amplify, uses the voice-assisted technology of Google’s AIY Voice Kit to provide gamified speech therapy for kids with cerebral palsy.

The team behind it is composed of Assistant Professor of Research Christopher Laine and Brian Cohn MS ’18, a National Science Foundation graduate fellow and student in the Computer Science PhD program at the USC Viterbi School of Engineering.

How does cerebral palsy affect speech?

Cerebral palsy, a congenital movement disorder that can result in a variety of movement and cognitive symptoms, stems from brain damage near the time of birth, often due to a stroke. One of the challenges patients face is the ability to communicate effectively, which is especially detrimental at an early age. “We believe no child should have his or her sense of expression suppressed,” Cohn said.

There’s a critical window where speech therapy is very important, early on when they’re children. And it might become less effective with age.

Christopher Laine

Team Amplify chose to assist those with cerebral palsy for two reasons: Laine’s area of expertise is oral-motor control; and those with the disorder often receive inadequate therapy. “There’s a critical window where speech therapy is very important, early on when they’re children,” Laine said. “And it might become less effective with age.”

Speech therapy for kids with cerebral palsy: Choose your adventure

Laine and Cohn sought to turn standard speech therapy on its head, making it more entertaining for children by creating a voice-assisted, choose-your-own-adventure game. “To move the story forward and interact with the characters, the child responds using voice therapy exercises,” Laine explained. The responses are recorded and evaluated on the spot. Depending on the response and therapy needs, the story continues in one direction or the other.

“For one exercise, patients have to make an extended ‘ah’ sound, so we had them get into a singing contest with the Phantom of the Opera,” Laine explains. “If the response was long enough, and they’re able to gain the respect of the phantom, he’ll help them in the next steps of the adventure. There’s another part where we wanted to do a modulation of voice tone, so patients have to distract a troll by making a questioning ‘Uhhhhh?’ sound to set their friends free.”

High-tech health care: Speech therapy for kids with cerebral palsy

Features such as how many exercises a patient has to complete before moving on to the next level can be fine-tuned on a case-by-case basis. Amplify would ideally be a device a child with cerebral palsy could take home to continue therapy. It would not eliminate the role of a speech therapist but would supplement traditional therapy. Following a session, a child’s “homework” can be recorded, analyzed and delivered to a speech therapist, providing additional information and helping to guide therapeutic strategies. Data can be retrieved via USB or uploaded immediately to the cloud.

“The therapist could even potentially interact with the device from his or her office and send updates,” Laine said. “The options are endless as far as being able to make it adaptive.” And since Amplify would enable patients to continue their exercises without having to go to multiple appointments, it could decrease costs.

With a take-home device that offers a variety of enthralling speech therapy adventures, children can practice/play whenever and wherever they choose.

Brian Cohn

“For some parents, a trip to the therapist with their child involves worrying about whether they can afford the gas, parking and time away from work,” Cohn said. “With a take-home device that offers a variety of enthralling speech therapy adventures, children can practice/play whenever and wherever they choose, opening up more time and saving more money for their family. And our prototype thus far can operate in homes without an Internet connection.” Additionally, Laine says there is a precedent for devices like Amplify to be covered by health insurance.

But will they like it?

While Amplify is still at a conceptual level and has not yet been tested on patients with cerebral palsy, Juan Espinoza MD ’10 at Children’s Hospital Los Angeles is helping to get the device into the hands of patients with cerebral palsy.

“We’re working on a prototype that will be deployed with a speech therapist at CHLA,” Laine said. “It would be a very limited trial to make sure the children interact with the device and enjoy it. If we have the content where we want it, the idea would be to have a set of patients take the device home.”

First of its kind speech therapy for kids with cerebral palsy

Currently, there aren’t any other voice-assistant devices on the market for speech therapy. Amplify will be the first. And once it is available, Laine can see it being applied to patient groups undergoing speech therapy for any reason.

“Gamified therapy is a trend that is gaining momentum,” Laine said. “I think voice assistants are going to be a logical next step for many applications.”

The idea of improving the quality of life for children with cerebral palsy and essentially giving them a voice is something that makes Laine and Cohn excited for the future.

“For anybody in a research field, the ultimate goal is to positively affect somebody’s life,” Laine says. “Sometimes these ideas are complicated, and sometimes they are simple. Amplify is a fairly simple concept, which can be implemented in many different ways. I hope that by virtue of its simplicity and flexibility, it becomes a reality fast.”

Amplify won first place — and $10,000 in prize money — in last year’s Voice Assistants for All Hackathon, hosted by the USC Center for Body Computing and the Working for Inclusive and Transformative Healthcare Foundation. Laine and Cohn have started the process of obtaining intellectual property rights for Amplify and have plans to incorporate and hopefully attract investors. The $10,000 in prize money will go toward creating a start-up company and product development for the prototype.

See video of the creators of Amplify talking about their hackathon win.