For some people with the neurological condition, synesthesia can be a lovely thing when the sounds they hear also produce colors.
But computer scientists from Tel Aviv University, the University of Pennsylvania and the University of Michigan have discovered a computer version of synesthesia that allows them to determine what’s being displayed on a monitor by listening to sounds emitted by the monitor.
As you might suspect, those sounds are very faint and not easily detected by human hearing. But they are there as tiny high-pitched tones produced by a monitor’s power supply in response to the varying demands of the screen display. But special equipment isn’t required to exploit the vulnerability that’s produced by this characteristic of most monitors.
Further reading
In their paper, which was released on Aug. 21, the scientists showed that those sounds could be recorded using the microphone in a standard consumer webcam, by a smartphone or by a digital assistant such as an Amazon Echo or Google Home device.
Furthermore, this exploit does not require the presence on site of the attacker. They can record the sounds over a remote call, such as one from Google Hangouts.
The amount of information that the researchers were able to discern using their method was remarkable. For example, they’re able to tell what web page is being displayed on the monitor during a web call, which means that they can tell if you’re paying attention to the screen that’s being shared or that you’re looking up baseball statistics.
Because eavesdroppers could tell what webpage you’re viewing and determine the text that is displayed, they could learn if you are filling out a form online and discover the personal information you are entering.
While it’s not clear if eavesdroppers can recover passwords this way if the contents of the password are obscured by dots or asterisks, the researchers did learn that they can read the contents of the typing on touch screen keyboards.
Touch screen keyboards have long been a favorite security tactic because they don’t make the typical sounds made as users strike each physical key, which can reveal what’s being typed. But the researchers found that they could discern contents typed on a touch screen keyboard because of the way it displays each letter as it’s typed with near-100 percent accuracy.
If that’s not enough to keep you up at night, the researchers showed that recordings made in the past can still reveal those sounds, which means that a recorded web call can reveal what was on your screen at the time of the call. But so can the contents of your Amazon or Google cloud files recorded by your digital assistants.
What’s happening, according to the research, is that some components in the power supplies of LCD monitors, which is nearly all flat-screen displays, emit faint sounds as the load on them changes. The electrical load changes in response to changes in the pixels displayed on the screen. So as the pixels change in response to computer inputs, the load on the power supply also changes and that results in a change to the sound being emitted.
The researchers were not able to determine which specific component was responsible for the sounds, and in fact, it may be more than one of them. However, they discovered that this vulnerability affects most monitors with 24-bit color depth, which is almost all of them. The text on the screen needs to be large enough to affect the power demands of the display, so apparently really tiny text probably can’t be read.
One of the reasons this worked is because the 60-Hz refresh rate on most monitors provides redundancy. So if the sound can’t initially be heard properly, there’s another chance every second.
The redundancy produced by the refresh rate on a monitor also makes it difficult to overcome the issue. I asked the researchers if you could solve the problem by producing sound of your own, such as by playing music.
“There is leakage emitted through high, inaudible frequencies, which playing music nearby is expected to have little effect on,” explained Roei Schuster, one of the authors of the paper in an email. “Such a measure alone would not provide any meaningful guarantee, and empirically the attack works well when recordings are made in a noisy office.”
Schuster said it might be possible to mask the noise produced by the monitor, but he explained in the paper that it could cause other problems, since some of the frequencies are within the range of human hearing.
However, it may be possible to block such sounds, he said. “The leakage can be attenuated using any sound-absorbing material. We expect that embedding such a countermeasure in commodity screens (and around screen-embedded microphones) would be expensive, however.”
When asked whether monitors with greater color depth, such as high-end 4K monitors, are also subject to the same vulnerability, Schuster said “it is yet to be verified that leakage exists in higher color depth screens, but 24 bit depth is used for almost all screens.”
Even when the researchers “experimented with wildly varying screen attributes, different manufacturer identities, resolution, production years (and decades), etc., we observed similar leakage across the board,” said Schuster.
So if you have a 4K monitor, you’re safe for now. But there’s no reason to think that such monitors don’t emit their own sounds, just that the researchers haven’t managed to decode them yet. But remember, this works on recordings, so if they find out later this exploit works with 4K monitors, than any recordings made in range of such monitors could reveal displayed information.
When asked if the exploit works with Microsoft Windows, Schuster said the researchers worked with the Ubuntu distribution of Linux, which has a cleaner screen appearance.
But he suggested that there is no reason to believe that Windows is any less vulnerable.
“In Windows, different displayed letters/websites will still have different and distinguishing acoustic emissions, and this is what our neural networks leverage,” Schuster said.
So what can you do if you routinely work with highly sensitive information? Other than not allowing any microphones into the same room with your computer, there’s not a lot. You can add a low-pass filter to the microphone you use for web calls, which will prevent it from hearing those sounds. You can make sure that smartphones and digital assistants are in another room. And you can make sure that there’s no sensitive information on your screen when you’re using Hang-outs or another VoIP application.
This situation is similar to a problem I worked with in the mid-1980s when I had to build a secure computer system for the U.S. Department of Defense. I had to prove that the emissions from the monitors I planned to use couldn’t be overheard by the Soviet surveillance ships parked off-shore. That was a different kind of noise, but the result would have been the same.
Today, this is an obvious exploit for a state-sponsored attacker. They will have what they need from this paper to exploit this vulnerability for cyber-surveillance. Now the next step is up to users with sensitive information. Now that they know it can be done, they can find a way to overcome it.
And if you need another reason to stay awake at night, two of the authors of this report published a paper in 2014 showing that it’s possible to steal cryptographic keys using acoustic noise.
