Baby Talk

A parent and toddler are playing together in a small room. Colorful blocks and plush toys lie scattered on the table between them. Mom smiles and coos; baby gurgles, claps and shakes her hands. But then mom turns her face away; she stops smiling and stops talking. She sits, expressionless and motionless, for two minutes.

“During this period, the baby is completely confused,” says Lauren Klein, a doctoral student in computer science at USC Viterbi School of Engineering. “The baby tries to do everything she can to get a response from her mom — through noises and gestures — and eventually gets frustrated. Finally, the pair start playing again, during a stage called the reunion.”

Klein is describing the “Still Face Experiment,” developed by American developmental psychologist Edward Tronick in 1975, which demonstrates how babies who are just learning about their relational world try to achieve connection.

More than 45 years later, this powerful test still provides insight into how a parent’s reactions can affect a baby’s emotional development. It’s one of the reasons that toxic stress — the body’s response to lasting and serious stress — can cause disruptions in the developing brain and body that can affect a child’s physical and mental health for a lifetime.

Providing support and intervention

Since 2019, Klein has been researching computer science and child development at the USC Interaction Lab, where she is supervised by Professor Maja Matarić.

Working with Dr. Pat Levitt and the researchers in his neurodevelopment lab at the Children’s Hospital Los Angeles, she is developing an automated system to model and support infant-caregiver interactions by detecting signs of toxic stress.

Described in two recent peer-reviewed conference papers for which Klein was the lead author, the system automatically analyzes videos of caregiver-baby interactions using techniques including artificial intelligence, machine learning and human-machine interaction.

The sooner we are able to detect a toxic stress response in infants, the sooner we can provide support and intervention to infant and caregiver.

Dr. Alma Gharib

Specifically, the system aims to evaluate nonverbal communication features — such as pose, body movement and facial expression — to help researchers learn more about how early interactions are shaped by the home environment. The team is also developing a mobile application to collect this data more easily, so that more families in difficult circumstances might receive assistance.

“The sooner we are able to detect a toxic stress response in infants, the sooner we can provide support and intervention to infant and caregiver, giving us the opportunity to support healthy development in every child,” says Dr. Alma Gharib, a clinical research director and staff scientist at the Levitt Lab who is working closely with Klein on the project.

Relationship dynamics

Not all stress is bad, but when a child experiences stress day after day — from maternal stress to witnessing abuse and extreme poverty — it can become what’s known as “toxic stress.” Children are especially vulnerable to this repeated stress activation, and previous research has found that toxic stress in itself can cause changes in infant-caregiver interaction. Improving relationships between the child and their primary caregiver, such as home visiting programs with health professionals that serve as coaches, can help to build responsive caregiving and alleviate the effect of toxic stress.

According to Amanda Perkins, a researcher at Vermont Technical College, up to 90% of children will experience at least one traumatic event, which can lead to toxic stress.

Up to 90% of children will experience at least one traumatic event, which can lead to toxic stress.

“Studying infant-caregiver interactions allows us to better understand stress-related health risk and conversely, resilience,” Gharib says. “Infant-caregiver interactions establish the primary framework for an infant’s social attention and development and their ability to make sense of the world. A loving and reciprocal relationship with a caregiver supports healthy development, often in the face of adverse experiences.”

Gharib emphasizes that “toxic stress response” refers to a prolonged activation of the stress response. “It’s not a response that is activated from a few minutes of inattention from the caregiver, but rather from frequent, persistent exposure to adversity, which the child’s system is unable to recover from,” she says.

But how can experts determine if a child is at risk? And what can caregiver-baby interactions tell us about the impact of toxic stress?

Complex human connection

One way is to invite parent and baby to the lab and record the pair playing to analyze specific aspects of their interaction. Then an expert can watch the videos and hand-code features of interest, such as gestures or tone of voice.

For instance, “serve and return interaction” is a common feature of play: Baby smiles and the caregiver counters with an engaging response. This positive form of interaction helps build neural connections in the baby’s brain that support communication and social skills.

But coding these metrics is time-consuming, labor intensive, and limits the number of insights that can be made and thus, how many families can be helped. Automated systems for coding caregiver-baby interaction do exist, but they tend to look at only one modality at a time — for instance, gesture — which gives an incomplete picture of the complex human connection that is occurring.

“Currently, someone would have to watch the video and code every few seconds for behaviors and affective states, or watch the entire video and provide measures,” Klein says. “To look at those interactions in more of a moment-by-moment basis or at a larger scale — for instance, for a huge study on toxic stress — is labor-intensive for research teams.”

Gathering new insights

Enter Klein’s AI-based system, which is capable of analyzing video interactions in a matter of minutes. The goal is not to immediately diagnose toxic stress, but to better understand caregiver-baby interaction and specific traits that may be indicators of toxic stress, such as difficulty sleeping or eating, or increased anxiety and aggression.

For instance, in recent studies, Klein and the team have looked at the pose and posture of mom and baby to better understand the relationship dynamics. In particular, they have been looking at responsiveness: How responsive does the parent seem to the child’s pose? How responsive is the child’s posture to their parent?

The system outputs an automated description of how parent and baby are interacting, along with a score. These insights can help guide experts in determining the best way to help a parent reduce their child’s stress levels, from talking and play to regular medical visits.

“We look at how they’re coordinating their postures with each other, or how they’re timing their nonverbal behaviors and gestures,” Klein says. “Essentially, we’re trying to develop measures of interaction dynamics to support the large-scale analysis of parent-baby interaction in the context of toxic stress. Can nonverbal communication such as head tilts and arm poses help us explore that?”

Working on computer science within a health care field allows me to make an impact in understanding how children develop and how adversity can affect child development.

Lauren Klein

Crucially, the system looks at multimodal communication — gesture, sound and physical proximity — to paint a more holistic picture of the interaction. Already, the team can measure infant-caregiver interaction using multiple behavioral cues at once, a long-standing challenge for completely automated approaches.

Right now, Klein is working with the researchers using known trends in child development to validate their measures. She aims to explore novel ways of understanding caregiver-baby interaction and help researchers conduct larger studies in the area of toxic stress.

“Working on computer science within a health care field allows me to make an impact in understanding how children develop and how adversity can affect child development,” Klein says. “And if we can understand it better by providing some automated analysis, I hope we can improve the scalability, which will allow the experts to help more families in need.”