Self-driving cars see better with cameras that mimic mantis shrimp vision

A new type of camera more clearly maps objects as the vehicle travels

EYE OF THE BEHOLDER Driverless cars could soon spot danger by viewing the world through polarized cameras that can take in a wide range of exposures at once, much like the eyes of mantis shrimps.

Viktor Gruev/Univ. of Illinois at Urbana-Champaign

By Jennifer Leman

October 12, 2018 at 7:00 am

To help self-driving cars drive safely, scientists are looking to an unlikely place: the sea.

A new type of camera inspired by the eyes of mantis shrimps could help autonomous vehicles better gauge their surroundings, researchers report October 11 in Optica. The camera — which detects polarized light, or light waves vibrating on a single plane — has roughly half a million sensors that each capture a wide range of light and dark spots within a single frame, somewhat similar to how mantis shrimps see the world.

The researchers wanted to “mimic the animals’ ability to detect a wide range of light intensities,” says coauthor Viktor Gruev, a bioengineer at the University of Illinois at Urbana-Champaign. The crustaceans’ visual system allows them to see both light and dark areas while moving in and out of dark crevices in shallow waters, he says.

The newly devised camera can take in a wider range of light intensities, measured in decibels, than other digital or polarization cameras. Previously, the best polarization cameras operated with a dynamic range of about 60 decibels; the new one works within a 140 decibel range, resulting in a clearer mapping of objects in the same frame.

Depending on the maker, autonomous vehicles currently use a mixture of methods to map the world around them, including lidar (light detection and ranging equipment), cameras and GPS. But the cameras currently guiding autonomous vehicles aren’t good at handling sharp lighting transitions and have trouble detecting features in foggy weather (SN: 12/24/16, p. 34). Because the new cameras are small and use many of the same parts as common digital cameras, Gruev says they could cost as little as $10.

Now you see me

Researchers recorded a street scene viewed through a digital camera that records in grayscale (left), which is a little better than the cameras currently used in self-driving cars, and through two polarization cameras. Those two views look slightly different because one shows the amount of light polarization (middle) and the other shows the angle of polarization (right). In the middle panel, red hues indicate high levels of polarized light and blue hues reflect low levels. In the panel on the right, red hues indicate horizontally polarized light and blue hues show vertically polarized light. Images from both polarization cameras pick up a higher level of detail than the grayscale camera, bringing other cars, pedestrians and bicyclists more clearly into view.