Photo courtesy of iStock

For the first time, new algorithms may be able to automatically explain why some self-driving cars crash, a question crucial to answer as more autonomous vehicles take to the roads.

According to a study by Tech Explore and King’s College London, the new approach reviews past events to explain why specific instances of failure happened, in the hope that this can be used to make improvements in the future.

Self-driving vehicles are increasingly being rolled out across the globe, in cities like London and San Francisco, but collisions and serious breaches of road safety have put pressure on manufacturers to explain why they make the mistakes they do.

This is often hard to do, and current methods only provide limited explanations for these.

Dr. Khen Elimelech, leader of the Autonomous Robots Lab at King’s and first author of the paper, said traditional methods rely on compiling failure statistics, to tell us how likely another failure is to happen in the future.

“But they cannot definitively tell you why a self-driving car made the specific error it did,” Elimelech said. “For that, you need to leverage what is known as actual causality, where an algorithm analyzes past mistakes retrospectively.”

Elimelech said the approach is particularly useful for self-driving cars where failures may stem from complex and rare causes and often have catastrophic implications.

Actual causality has previously only been trialed in AI used to classify images.

It is the first time this concept has been applied to more complicated case of AI-driven cyber-physical systems, Elimelech explains.

According to Tech Explore, understanding exactly which events explain the crash is a challenge and has acted as a barrier to deployment in the past.

The current approach builds on previous work from the team, which introduced a novel algorithm to efficiently and proactively identify those rare scenarios that would result in a crash, a problem called “falsification.”

The group’s new work takes it further, analyzing the crash scenarios found through falsification, in order to explain them.
It does so by sifting through all the potential causes of a crash to pinpoint the root cause of the failure.

Yet, finding these root causes is not easy. Autonomous vehicles operating in the real-world must continually process observations of other objects, humans, and cars around it to make driving decisions.

It means that when a crash happens, the number of potential causes that could have led to it are huge.

In some cases, an object that the car saw on the road miles before a crash, can be what started a chain of events that ultimately led to the collision.

To address this, the work also includes a practical “responsibility-guided” search algorithm, capable of quickly sifting through all the potential causes.

The algorithm is capable of returning an explanation for an event with orders of magnitude less computational effort than the baseline algorithm.

Elimelech added, “In a world where autonomous vehicles are taking up more space on London’s streets, being able to explain why something happened is vital if we’re going to build trust with this type of technology and integrate cyber physical systems like this into our lives.”

In the future, the team hopes to develop algorithms that can support even more complex applications, such as the potential introduction of autonomous assistive robots in care homes, to help design systems across a broad range of domains that are reliable and explainable, paving their way to future integration into society.