We all know the score when it comes to surveillance in TV & Movies these days.
Scene: The killer has all but got away with it until some upstart agent spots a smudge in a reflection…
Agent: “Can you guys zoom in on that region”
Agent: “Now enhance it a little”
Tech: “How’s that?”
Agent: “…a little bit more…bingo! We’ve got him!”
If you have no idea what I’m talking about pop over to zoomandenhance.tumblr.com for some entertaining examples.
To get to the point, I set myself the challenge of being able to reconstruct an image from the seemingly impossible. A task partly inspired by this work…
Instead of glitter I would use crumpled foil.
My set up was to use a laptop screen to display images, the crumpled foil placed on the keyboard, the laptop screen tilted forwards and the camera pointed at the foil reflecting the images on the screen.
First stage to reconstructing the images is to profile the foil in terms of a reflectance map. To do this I display a black screen with a white rectangle in different known x-y positions. Because I’m in control of the white test square I can capture a frame, from the camera, of the crumpled foil with the test square at different grid positions on the laptop screen. This gives me, in this case, a 20×20 matrix of matrices. Each sub matrix is a 1280×960 grayscale image, this is the image captured from the camera that’s pointed at the crumpled foil. The 20×20 matrix is the position of the white rectangle on the laptop screen. This video shows the output from the camera as the white test square scans across the laptop screen.
After the calibration I now know what parts of the reflectance image, from the foil, contribute to what parts of the laptop screen. The theory now is that if I display an image on the screen, capture the reflected image from the crumpled foil I should be able to back project this mess below in order to partially reconstruct the input image, but only to an accuracy of a 20×20 image as that’s the accuracy used for calibration. If I use a finer calibration matrix, say 100×100, the white square on the laptop screen is also reduced in size therefore it doesn’t provide enough illumination to overcome the background light that leaks through the black regions of the screen. The black regions of an LCD are still illuminated from behind, they are not completely dark.