Video game promotes social engagement for children with autism

An autistic child fires up an iPad and launches an app for Social Clues, a video game with a higher purpose. Developed by a team of 35 USC students, including engineering, MBA, design and others, the game transports players to a make-believe world that entertains, while teaching children on the spectrum to make eye contact, listen to others and engage in conversation.

In the game, a child assumes the identity of particiPETE or communiKATE. Pete and Kate converse with different virtual characters in cafeterias, classrooms and other real-world settings to find lost toys. In the process, players learn the dos and don’ts of social interaction.

In a virtual cafeteria, for instance, a computer-animated figure named Pete meets Max, a wide-eyed little boy who becomes frustrated when a vending machine runs out of his favorite beverage.

“I want apple juice,” Max yells. To prepare Pete for conversation, Sherlock, a colorful, friendly parrot that gently guides players to the right choices, encourages Pete to drag an arrow so that his eyes meet Max’s. “When we talk to somebody,” Sherlock says, “we should look at them and make eye contact.”

Next, four virtual faces appear on screen in an exercise designed to improve a player’s emotional awareness. Sherlock asks which expression best reflects Max’s feelings. If Pete chooses the sad face, the parrot says, “Max’s reaction is stronger than sad. Let’s try another.” (The correct answer is angry.)

“What we’re trying to do is break down everyday interactions into something very understandable, very manageable,” said USC Marshall School of Business MBA student Jeremy Bernstein, the game’s project lead, who created it with his wife, Karen Okrent, a speech pathologist who often works with children with autism. “We’re basically giving our players a road map they can use offline.”

A need for new tools

Although easy to play, complex engineering undergirds Social Clues. To prevent the app from crashing, for example, five USC Viterbi School of Engineering students have worked to reduce the memory demands placed on iPads and other devices. They have done so by imperceptibly shrinking the size of images and limiting the number of game components without compromising quality, said lead engineer Fotos Frangoudes, a USC Viterbi computer science PhD student.

“I make sure all the ideas we have can be implemented and that everything’s on time,” said Frangoudes, whose research interests include health apps in games.

With an estimated one of 68 children identified as having autism spectrum disorder, a neurological condition that impairs communication, the ability to form relationships and respond appropriately to one’s surroundings, according to the Centers for Disease Control and Prevention, the need for new tools such as Social Clues to treat this incurable disorder has never been greater.

In producing the game, developers held more than a dozen working sessions with autistic children, therapists and others. Feedback from these groups led to changes in the user interface for clarity, including bright colors, simple characters, big buttons and easy-to-read icons. Game analytics provide parents and therapists with important information to better understand areas where an autistic child might need more attention.

Social Clues came to life thanks to USC’s “Advanced Games” course, a yearlong class that nurtures games from idea to completion. In the fall 2013 and this spring’s semesters, team members worked into the night to develop a prototype. They have continued to put in long hours since to produce a potential commercial version, which Bernstein said could be available in the near future.

The game’s burgeoning success is but one of many spawned by “Advanced Games.” Now in its ninth year, this jointly run class from USC Viterbi and the USC School of Cinematic Arts has attracted aspiring gamers from all over the country. It has also launched several hot games, including The Adventures of P.B. Winterbottom, and served as a talent incubator for leading video game industry programmers and software engineers.

With 250 participants, the class is believed to be one of Los Angeles’ five biggest game-development studios, said Michael Zyda, professor of the course and a USC Viterbi computer scientist. It has also helped make the USC Games program the Princeton Review’s No. 1-ranked graduate game design program for five consecutive years.

“Our students are pretty awesome and build great games,” Zyda said. “Social Clues is one of them.”