New imaging tech helps ordinary cameras take out-of-sight pictures | Science News

Support Science Journalism

Science News is a nonprofit.

Support us by subscribing now.


Ordinary cameras can now photograph out-of-sight objects

The photo-analyzing system works similar to a periscope mirror

1:00pm, January 23, 2019
reconstructed images

I SPY  By analyzing a multicolored jumble of light and shadow on a wall (middle), a new computer program can create a rough reconstruction (right) of an image displayed on an out-of-frame LCD screen (left).

Sponsor Message

With a new photo-analyzing computer program, a photographer can take a picture of something that’s not even in frame.

The system analyzes light that’s reflected off matte surfaces, such as walls, to discern out-of-sight images, similar to the way a periscope mirror reveals what’s around a corner. Whereas other techniques for spotting out-of-sight objects require expensive, specialist optical equipment (SN: 1/9/16, p. 15), the new program can render a rough, full-color reconstruction of a hidden scene using a single snapshot captured with an ordinary digital camera.

“Looking way beyond what the camera can see could be very, very useful” for self-driving cars monitoring surrounding traffic or doctors using endoscopes to probe patients’ bodies, says Aswin Sankaranarayanan, a computational photography researcher at Carnegie Mellon University in Pittsburgh not involved in the work. This technology, described in the Jan. 24 Nature, may also help police monitor buildings from the outside during hostage situations or first responders scout out collapsed buildings after disasters.

Electrical engineer Vivek Goyal and colleagues at Boston University tested their system by displaying images on an LCD monitor facing a wall. In between the two, the researchers placed an object that blocked some of the light emanating from the monitor, casting shadows on the wall. (In this case, Goyal’s team used a rectangular panel, but an obscuring object of any size or shape would do.)

Both the light that reaches the wall and the shadowing created by the intervening object contain clues about the LCD display. To picture why, imagine a screen displaying two white circles side by side against a black backdrop. Someone looking at the smear of light that these circles collectively cast on the opposite wall may not be able to tell whether the screen displays one or several bright spots. But put a chair between the monitor and wall, and that chair will cast two distinct shadows, making it clear that the screen includes two distinct sources of illumination.

Sight unseen

New photo-analyzing software can scrutinize the pattern of light reflected off a wall to discern an image displayed on an LCD screen in the same room. 

Using this kind of analysis, the new computer program scrutinized the pattern of light in a photo of the wall to roughly reconstruct the image displayed on-screen. The system’s re-creations of pictures like cartoon faces, letters and striped patterns “are really quite impressive,” says Genevieve Gariepy, an engineering physicist not involved in the work.

The technique’s reliance on having an object in between the scene of interest and photographed surface may limit its practical use, says Gariepy, who developed technology to track objects around corners at Heriot-Watt University in Edinburgh. But many cluttered environments, like rooms and roads, are likely to have obscuring objects aplenty, she says, so “I don’t think that’s a showstopper” for the new imaging scheme.


C. Saunders, J. Murray-Bruce and V.K. Goyal. Computational periscopy with an ordinary digital camera. Nature. Vol. 565, January 24, 2019, p. 472. doi:10.1038/s41586-018-0868-6.

Further Reading

M. Laurenzis. Shadows used to peer around corners. Nature. Vol. 565, January 24, 2019, p. 435. doi:10.1038/d41586-019-00174-1.

A. Grant. There’s no hiding from new camera. Science News. Vol. 189, January 9, 2016, p. 15.

Get Science News headlines by e-mail.

More from Science News

From the Nature Index Paid Content