A bag of chips sits, apparently silently, on camera. Offscreen, music plays, creating a ripple of nearly undetectable vibrations across the bag’s thin foil layer. In video playback, computer scientists from MIT study the bag’s tiny movements and extract from it the audio that made it sway.
According to the group of researchers from Adobe, Microsoft and MIT, the experiment — called “The Visual Microphone” — aims to “recover sounds from highspeed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound.”
The scientists plan to publicly release their code that enables them to record these tiny vibrations.