Ever wonder what the people on the other end of a Hangouts session are really looking at on their screens? With a little help from machine learning, you might be able to take a peek over their shoulders, based on research published at the CRYPTO 2018 conference in Santa Barbara last week. All you’ll need to do is process the audio picked up by their microphones.
Daniel Genkin of the University of Michigan, Mihir Pattani of the University of Pennsylvania, Roei Schuster of Cornell Tech and Tel Aviv University, and Eran Tromer of Tel Aviv University and Columbia University investigated a potential new avenue of remote surveillance that they have dubbed “Synesthesia“: a side-channel attack that can reveal the contents of a remote screen, providing access to potentially sensitive information based solely on “content-dependent acoustic leakage from LCD screens.”
The research, supported by the Check Point Institute for Information Security at Tel Aviv University (of which Schuster and Tromer are members) and funded in part by the Defense Advanced Research Projects Agency, examined what amounts to an acoustic form of Van Eck phreaking. While Van Eck phreaking uses radio signal emissions that leak from display connectors, the Synesthesia research leverages “coil whine,” the audio emissions from transformers and other electronic components powering a device’s LCD display.