With artificial intelligence, common sense is uncommon

Common sense isn’t common, especially when it comes to artificial intelligence. Computers struggle to make fine distinctions that people take for granted. This is why websites require you authenticate your humanity before logging in or making a purchase: Most bots can’t tell the difference between a crosswalk and a zebra.

At the USC AI Futures Symposium on AI with Common Sense earlier this month, more than 20 USC researchers reported on the technical reasons why that’s the case, and different avenues of research to address this. Advances in common sense AI will improve human-facing services, from enhanced social services to better serve society to personal assistants that better predict our context and needs.

“AI systems today can converse with us to order a book, find a song, or vacuum our floors,” said Yannis Yortsos, dean of the USC Viterbi School of Engineering. “But they do not have the common sense to know that we read books for learning and for pleasure, that music relaxes us, and that tidy homes are more enjoyable. Mindsets taking into account human interaction must be applied in tackling the commonsense challenge for AI as we are laying the foundations for AI to be responsible and ethical, and to impact society in meaningful ways.”

AI still makes ‘silly mistakes’

Today’s AI systems can’t make presumptions about situations or information that people encounter daily. Your phone’s camera for instance, reads the visual information in frame and focuses on a particular subject utilizing AI. However, differentiating between a white shirt and a white wall can cause AI to fail because it doesn’t recognize the other differences between a shirt and wall, only the color.

To help overcome this challenge, researchers use several sources of commonplace knowledge like Wikidata to obtain a “reasoned” AI response. Filip Ilievski, research scientist at USC’s Information Sciences Institute (ISI) and organizer of the symposium, has developed an AI-based program using multiple sources of commonsense knowledge to complete a human-initiated story. For instance, a user might type in, “I am at home and I want to warm up but there is no blanket” and the AI would reply, “Use a jacket.”

We keep finding one of the key obstacles preventing us from integrating AI capabilities is the lack of common sense.

Filip Ilievski, USC’s Information Sciences Institute

“We keep finding one of the key obstacles preventing us from integrating AI capabilities is the lack of common sense,” he said. “On one hand we have AI that is capable of very impressive things but at the same time, we have AI that makes silly mistakes. Currently, we tend to build one AI agent per task. We want to have comprehensive commonsense knowledge sources that allow AI agents to perform well on many tasks.”

Expert input, crowdsourcing and extracting from large amounts of text are a few of the approaches that researchers use to support commonsense reasoning. These various knowledge sources are especially useful when faced with incomplete information. By using everyday assumptions in their logic, AI agents can make educated assumptions for familiar as well as unexpected situations.

“We typically think of common sense as something you expect another adult to know or things that let us know how to interact and interpret the world around us,” said Marjorie Freedman, research team lead at ISI. “AI needs common sense to accurately interpret the world and serve in a useful collaborative capacity. Depending on what aspect of common sense you’re trying to learn and how you’re looking to use that information, AI might use crowdsourced data to augment that knowledge automatically.”

Creativity driving innovation in AI robots and agents

With a comprehensive knowledge base, AI can then develop novel ideas and approaches through computational thinking and creativity. Mayank Kejriwal, research assistant professor of Industrial and Systems Engineering and a research lead at ISI, is investigating what features a computational model requires to effectively produce ideas.

“We’re in a very exciting time for AI creativity,” Kejriwal said. “A recent project using AI allowed mathematicians to provide an idea that might seem unintuitive initially, but then it turns out they can solve these very complicated math theorems where AI gives the idea of how to solve it. And despite these advances, there are still very simple things people are able to do but AI struggles with such as determining whether two things are the same or different. There is still a disconnect in what we can intuitively do and what AI can intuitively do.”

A challenge for AI is reading emotions. Jonathan Gratch, research professor in Computer Science and Psychology and director for Virtual Humans Research at the USC Institute for Creative Technologies, created a model that adds situational awareness to the facial recognition techniques currently used in AI to recognize an emotion. By doing so, AI can begin to understand people’s goals and model an appropriate reaction to a particular emotion.

AI hasn’t tended to deal with emotions until quite recently, but it is inescapable when you have to deal with human behavior.

Jonathan Gratch, USC Institute for Creative Technologies

“AI hasn’t tended to deal with emotions until quite recently, but it is inescapable when you have to deal with human behavior,” Gratch said. “It would be great if machines could recognize and understand how people or groups feel and then also forecast and shape the downstream consequences of those feelings. The tricky thing is much of what determines a person’s emotional response is hidden.”

Understanding human motivation remains a principal challenge in commonsense AI, and the work at USC integrating AI research with research in social sciences like cognitive science or psychology leads to better approaches, according to Yolanda Gil, research professor in computer science and senior director of strategic initiatives in artificial intelligence and data science at ISI. “This crucial area of research will drive innovation in AI, and USC researchers will be leading the way,” she said.

“USC and ISI are doing amazing research in AI,” said Bart Selman, president of the Association for the Advancement of Artificial Intelligence and professor of computer science at Cornell University. “The research taking place goes to the core of the open challenges in AI in common sense, knowledge and reasoning.”

USC researchers are already using common sense in AI systems for many applications, including understanding cultural differences, helping children with autism, assisting with household tasks and detecting biases in news.