Tech Can Push Us Apart, but These Games and VR Build Empathy

Jennifer Flores welcomes visitors into her home — a tent in L.A.’s Echo Park. She points out the makeshift features of her small living space and explains the challenges of daily life with little privacy, no electricity and a constant state of heightened alertness. She shares the story of her life, her kids, her struggle to find work and more.

The tent is a tight fit for one, yet thousands of visitors from across the world have been inside. They joined her via an immersive 360-degree video. The 3D augmented reality experience uses the social media app Snapchat to place Flores’ tent within the viewer’s own room, complete with narrated annotations of her home.

“You can actually step inside her tent, crouch down and hold presence with her home — the bedroom, the kitchen, her living room — all in that tent,” says Robert Hernandez, professor of professional practice at the USC Annenberg School for Communication and Journalism. “And it just shows you so many details. It’s a new way of storytelling.”

The students ask, ‘How might you use this technology to tell your story? What’s the story from your perspective that you would tell the rest of the world?’ We’re not going to decide the story; we’re going to help you develop and produce that story.

Robert Hernandez

Flores’ tour of her home is a part of Homeless Realities, a project from a course led by Hernandez in which USC students use virtual reality and other emerging technologies to tell stories that might be otherwise overlooked by traditional journalism. The class, called “JOVRNALISM,” uses innovative technology to empower people from marginalized communities. Recent projects enabled foster youth and domestic abuse survivors to share their voices.

Hernandez explains that his students talk to people and show them their digital tools. “The students ask, ‘How might you use this technology to tell your story? What’s the story from your perspective that you would tell the rest of the world?’ We’re not going to decide the story; we’re going to help you develop and produce that story,” he says.

At a time when tech could be blamed for causing division and polarization, this technology affects empathy differently: Its whole purpose is to build understanding. And the USC Annenberg students aren’t alone in their empathetic quest. Across USC, students and educators are developing and using technology to tap into different perspectives, challenge assumptions and place us in the shoes of people leading much different lives.

How Does Technology Affect Empathy? These Games Encourage Feeling

From the first 8-bit programs on, video game players have found new worlds through their screens. But games and interactive media can also introduce users to unknown worlds that are just a few blocks away in real life. Game designers are using their media for more than just an escape — in some cases, games can make players more aware of a reality they don’t know.

“If a game developer can immerse the player in a story that makes the player feel like they’re peering into the window of someone else’s life, and that experience is outside of the player’s comfort zone, that’s where the magic happens and the light switches on,” says Jim Huntley, professor in the Interactive Media and Games Division of the USC School of Cinematic Arts and head of marketing for USC Games. In games and interactive media, players are responsible for the decisions they make through the adventure. And if the gameplay decision involves a moral dilemma, Huntley says, it can shape how immersive a game feels and how much ownership a player feels toward the experience.

As a student in the USC Games program, Bryant Young ’17, MS ’20 took that lesson to heart. In his game Our America, he recreated a real-world experience using virtual reality: He put players in the role of a Black father driving his son to school. As a police officer pulls the car over, the player must decide how to act. Reaching for the glove compartment or making another move at the wrong time could end in tragedy. Young’s capstone project offers players a small window into the pain and fear that Black people face every day, he says. “Honestly, my goal is to make people cry.”

Young, now co-founder and CEO of Fishean Studios, aims to expand and release Our America, which has received industry acclaim and grant funding from game companies including Oculus and Unity. He wants Our America to be the start of a series of virtual reality experiences that bring stories from marginalized communities to life.

“People will look at anything with a controller and say it’s a game. And they’re like, ‘Oh, you can’t get talking about serious stuff because it’s a game. You can’t be political because it’s a game,’” Young says. “But the thing is, interactive media and VR can make more than just games. They can make experiences, they can make stories, they can have choices that can be impactful.”

Marientina Gotsis, director of the USC Creative Media and Behavioral Health Center and founder of the USC Games for Health Initiative, agrees that designers can offer a message beyond just the fun of gameplay. But the USC School of Cinematic Arts associate professor notes that simply providing an immersive, realistic experience might not be enough to prompt someone to embrace new beliefs and behaviors.

“We have good psychological reasons for not caring too much about other people, and there is a lot of research on this. People can shut down while feeling other people’s pain; they can be traumatized, they can be hardened,” Gotsis says. “VR and interactive media’s capacity to promote empathy in action — or compassion — can be leveraged by good storytelling and modeling how to balance self-care and caring for others.”

Compassion is a limited resource that we all have, regardless of what our job is or what responsibilities we have in our personal or professional life.

Francesca Palamara

Gotsis was able to see this in action as an advisor to USC Games alumna Francesca Palamara MFA ’21, who designed Alma, a game about a day in the life of a Cuban immigrant. The player takes on the role of a mother and nurse in a Florida children’s hospital. She must balance her work, family and personal life while keeping an eye on a limited “compassion meter” throughout the nurse’s busy day. Alma is a simple but powerful game that illustrates the difficult balance that health care workers must strike.

As background research for the game, Palamara interviewed pediatric nurses about their jobs and lives. COVID-19 brought the issue into even sharper focus, she says, adding that nursing programs have shown interest in using Alma as a training tool.

“Compassion is a limited resource that we all have, regardless of what our job is or what responsibilities we have in our personal or professional life,” she says. “And I think that’s the power of games — being able to create that dialogue, that connection with someone far, far away.”

Empathic and Human by Design

Could virtual reality help us understand the struggles of someone we haven’t met yet? Albert “Skip” Rizzo thinks so. The director of medical virtual reality at the USC Institute for Creative Technologies (ICT) uses virtual characters to help health care workers learn to talk with patients. These virtual humans developed at ICT — lifelike 3D avatars that move and speak like real people — can be programmed to act like a patient with a medical condition like post-traumatic stress disorder or depression.

In a 2020 study, Rizzo and his colleagues found that patient interaction skills were significantly better in health care workers who trained with virtual patients compared to those who only received traditional lectures and assigned readings. Workers who completed the virtual training were better at “motivational interviewing” — a counseling approach that aims to help patients change their health habits through a collaborative discussion rather than just throwing scientific data at them. Rizzo, a research professor at the USC Leonard Davis School of Gerontology and Keck School of Medicine of USC’s Department of Psychiatry and Behavioral Sciences, is also investigating the use of virtual humans in other communication skills training. The technology could provide opportunities to practice communicating in stressful situations such as a job interview, as well as helping military veterans face post-traumatic stress disorder and people on the autism spectrum manage social anxiety.

ICT’s advanced technology is also preserving the stories of survivors of traumatic experiences and sharing their accounts with audiences that can learn from them. The Digital Survivor of Sexual Assault project records testimonials in the center’s groundbreaking Light Stage system, a spherical stage mounted with lights and cameras powerful enough to capture every angle of a subject’s face and body. Researchers can create 3D avatars of the survivor so audiences can hear a story told “firsthand,” ask questions and learn just as they would with an in-person speaker.

This “face-to-face” way of sharing the stories of those who have survived atrocities can foster understanding and effectively counter hatred that might otherwise lead to violence and genocide, says Kori Street, senior director of programs and operations at USC Shoah Foundation—The Institute for Visual History and Education. The foundation has collected more than 55,000 video testimonies of survivors and witnesses of the Holocaust and genocides around the world to preserve their narratives for future generations.

The foundation worked with ICT to bring these stories to life in engaging and illuminating formats. The resulting Dimensions in Testimony technology allows people to virtually meet witnesses of the Holocaust and other genocides. Using AI-powered interactive recordings, the audience can hear real-time, conversational answers to their questions from a 3D projection. Street notes that afterward, many participants talk about their experience as if it was a real-life conversation.

“Participants often forget that it’s technology they’re working with, that they’re interacting with a recorded testimony,” she says. “They’re focused on the conversation and their curiosity. … It really is connecting you as a human being to another human being and their experience and history.”

They’re focused on the conversation and their curiosity. … It really is connecting you as a human being to another human being and their experience and history.

Kori Street

Embracing technology has allowed the institute to bring testimonials to people in many different ways, from augmented reality presentations at historic sites to a web portal tailored for young students. “The thing that we have always done at Shoah Foundation is leverage technology to help us not only capture the survivor stories but to also take those stories where our users — scholars, educators, learners — are,” Street says. Research has shown that sharing genocide survivors’ stories — especially in interactive formats that make them come alive — increases participants’ engagement in learning, understanding of the historical events and willingness to stand up for people who are different than them.

As a journalism professor specializing in emerging technology, Hernandez understands tech’s power to transform stories. But he points out that while digital tools have huge and exciting implications, they’re just tools in a journalist’s or storyteller’s toolbox. Amplifying people’s stories is the point.

He predicts that virtual and mixed reality technology will usher in societal shifts on the same scale as the monumental changes that came with the rise of the internet. With cultural impacts ahead, he believes that he and his students have the responsibility to use technology to empower and bring people together rather than divide them.

“What I hope is that we have helped shape the framework around how to use this technology ethically and to serve communities,” Hernandez says. “It could be used for ill, like deepfakes. Or it could be used for really wonderful moments to hold presence with another person.”