relation: http://miis.maths.ox.ac.uk/miis/749/
title: Measuring Vibrations from Video Feeds
creator: Brambley, Ed
creator: Fletcher, Helen
creator: Hill, Roger
creator: Johnston, Ifan
creator: Liu, Jessie
creator: MacKay, Robert
creator: Mathews, J.
creator: Ockendon, John
creator: Piette, Bernard
subject: Aerospace and defence
description: By using a high-speed camera, researchers at MIT in 2014 where able to recover human speech from videos of minute vibrations of objects in a room. For example, in one experiment a 2,200fps camera was positioned outside a room behind sound-proof glass, videoing an empty crisp packet on the floor inside the room, while a researcher shouted “Mary had a little lamb” at the crisp packet. By detecting minute oscillations of the crisp packet of 1 μm (0.001 mm), and using hours of computer processing, a ten second audio clip could be produced that was recognisably “Mary had a little lamb” in an American accent.  The purpose of this study group was to investigate whether this tech- nique could be used in practice, with emphasis on the recovery of intel- ligible speech from a video feed of a room. During the week, the group investigated several aspects of the problem, including:  • how much an object vibrates due to sound;  • what can be done to maximize the vibration;  • how the MIT technique detects minute vibrations in videos; • what affects the quality of the resulting recording; and  • how good a recording is needed for intelligible speech.  It was discovered the MIT experiments would not have recovered intel- ligible speech from an ordinary conversation; their success depended on loud sounds and prior knowledge of “Mary had a little lamb”. Camera vibrations were also ignored by MIT; these are expected to be signifi- cant, but the technique could be adapted to be resilient to them. Other possibilities for enhancing their technique, by exploiting resonances or reflections, are discussed in the report. A high-speed low-noise cam- era is essential, and any existing video footage (such as from CCTV) is unlikely to be of sufficient quality. Further experiments with high-end high-speed cameras are needed to assess the feasibility of the technique in practice.
date: 2017
type: Study Group Report
type: NonPeerReviewed
format: application/pdf
language: en
identifier: http://miis.maths.ox.ac.uk/miis/749/1/mod_report.pdf
identifier:   Brambley, Ed and Fletcher, Helen and Hill, Roger and Johnston, Ifan and Liu, Jessie and MacKay, Robert and Mathews, J. and Ockendon, John and Piette, Bernard  (2017) Measuring Vibrations from Video Feeds.  [Study Group Report]