Ordinary cameras can now photograph out-of-sight objects

With a new photo-analyzing computer program, a photographer can take a picture of something that’s not even in frame.

The system analyzes light that’s reflected off matte surfaces, such as walls, to discern out-of-sight images, similar to the way a periscope mirror reveals what’s around a corner. Whereas other techniques for spotting out-of-sight objects require expensive, specialist optical equipment (SN: 1/9/16, p. 15), the new program can render a rough, full-color reconstruction of a hidden scene using a single snapshot captured with an ordinary digital camera.

“Looking way beyond what the camera can see could be very, very useful” for self-driving cars monitoring surrounding traffic or doctors using endoscopes to probe patients’ bodies, says Aswin Sankaranarayanan, a computational photography researcher at Carnegie Mellon University in Pittsburgh not involved in the work. This technology, described in the Jan. 24 Nature, may also help police monitor buildings from the outside during hostage situations or first responders scout out collapsed buildings after disasters.

Electrical engineer Vivek Goyal and colleagues at Boston University tested their system by displaying images on an LCD monitor facing a wall. In between the two, the researchers placed an object that blocked some of the light emanating from the monitor, casting shadows on the wall. (In this case, Goyal’s team used a rectangular panel, but an obscuring object of any size or shape would do.)

Both the light that reaches the wall and the shadowing created by the intervening object contain clues about the LCD display. To picture why, imagine a screen displaying two white circles side by side against a black backdrop. Someone looking at the smear of light that these circles collectively cast on the opposite wall may not be able to tell whether the screen displays one or several bright spots. But put a chair between the monitor and wall, and that chair will cast two distinct shadows, making it clear that the screen includes two distinct sources of illumination.

Using this kind of analysis, the new computer program scrutinized the pattern of light in a photo of the wall to roughly reconstruct the image displayed on-screen. The system’s re-creations of pictures like cartoon faces, letters and striped patterns “are really quite impressive,” says Genevieve Gariepy, an engineering physicist not involved in the work.

The technique’s reliance on having an object in between the scene of interest and photographed surface may limit its practical use, says Gariepy, who developed technology to track objects around corners at Heriot-Watt University in Edinburgh. But many cluttered environments, like rooms and roads, are likely to have obscuring objects aplenty, she says, so “I don’t think that’s a showstopper” for the new imaging scheme.