Leaking Screen Information on Zoom Calls through Reflections in Eyeglasses

Okay, it’s an obscure threat. But people are researching it:

Our models and experimental results in a controlled lab setting show it is possible to reconstruct and recognize with over 75 percent accuracy on-screen texts that have heights as small as 10 mm with a 720p webcam.” That corresponds to 28 pt, a font size commonly used for headings and small headlines.

[…]

Being able to read reflected headline-size text isn’t quite the privacy and security problem of being able to read smaller 9 to 12 pt fonts. But this technique is expected to provide access to smaller font sizes as high-resolution webcams become more common.

“We found future 4k cameras will be able to peek at most header texts on almost all websites and some text documents,” said Long.

[…]

A variety of factors can affect the legibility of text reflected in a video conference participant’s glasses. These include reflectance based on the meeting participant’s skin color, environmental light intensity, screen brightness, the contrast of the text with the webpage or application background, and the characteristics of eyeglass lenses. Consequently, not every glasses-wearing person will necessarily provide adversaries with reflected screen sharing.

With regard to potential mitigations, the boffins say that Zoom already provides a video filter in its Background and Effects settings menu that consists of reflection-blocking opaque cartoon glasses. Skype and Google Meet lack that defense.

Research paper.

Tags: , ,

Posted on September 23, 2022 at 6:43 AM • 25 Comments

Comments

Four Eyes • September 23, 2022 7:40 AM

In general I find glasses reflections annoying, and yes, leaking private info.

Anyone have recommendations on glasses options/features to reduce this? (i.e. does anti-glare for me also reduce 3rd party reflections?)

SM • September 23, 2022 8:02 AM

@Four Eyes
More transparent glass should reflect less light.
However, the best option would be to have a polarizer in the camera, also using dark themes should reduce the amount of glare of your screen, combined with better lighting for you that does not illuminate your screen should do the trick

Brandt • September 23, 2022 8:27 AM

For many people it’s a very real threat! A lot of legal proceedings that moved online during the pandemic have remained there. And a lot of attorneys have upgraded their work from home setups, including 4K cameras. Hearings that previously occurred in a courtroom using paper documents now take place on Zoom with on-screen documents, so those eyeglass reflections are being live-streamed to adversarial counsel. The attack may be impractical today, but it will soon be packaged up and made easy to use.

JonKnowsNothing • September 23, 2022 10:58 AM

@All

For those of us staring at computer screens for long hours, our eyesight gets slammed and difficulties can develop in our vision.

Computer vision syndrome is a catch all for the many types of eye issues that people develop from constant staring at the screen, brightness, ambient lighting, contrast, eye muscle fatigue, tearing, dry eyes, painful headaches and more.

Programming is a medically hazardous profession.

Being an end user is worse.

They get all the same no-benefits and also no say in screen UI. Recent additions of skins, colors, tones and background may help some but the staring part just continues for longer as some of these mitigations only delay the onset of STOP.

There are some companies that sell specialty gamer eye glasses, which are more like indoor sunglasses. Some come with different attributes like magnification levels so you don’t have to squint while playing on a laptop. I’ve not used them personally, but anecdotal reports from other gamers are they can help a lot.

I haven’t heard of issues with re-reflections of screen elements during video gaming tournaments but that’s a new twist to consider.

There’s also the old-new-again Glass-oles coming w displays and cameras embedded in the frames, you won’t need much reflection if the frame transmits what’s in front of the pin hole camera.

The Spoon Sequence with Reflections in The Matrix was a fun section and nicely done. It gets repeated as a motif in other parts of the series.

  • There is no spoon…

Quantry • September 23, 2022 11:05 AM

Huh.

The h2 heading for this article is “font-size: 24px;” near as I can tell.

75% accurate? Geez, don’t wet the bed yet.

  • Wear a few bright LEDs on yer head band, especially including in the IR and UV wavelengths if you think visible ones are too unprofessional.
  • Webmasters make yer website headings smaller.

Again, I have a feeling you have MUCH more pressing problems since that kind of money is being thrown up against you. And it is; an ocean of money in fact, and an endless sea of obsessed voyeurs who justify “brownie points at any cost”.

“Accept this, and all else.”

My question: who knows a basic circuit for strobing these LEDs using the random behavior exhibited when current sweeping a signal diode in the quantum tunnelling region? $2 budget.

h t t p s ://www.nature.com/articles/s41598-017-18161-9

Anonymous Coward • September 23, 2022 11:57 AM

A few different thoughts.

Perhaps an attorney could “leak” specially-prepared text to an adversary.

What if your boss or customer sees you watching a “private” video during a meeting? (You can assign different values to “private”.)

Ted • September 23, 2022 1:23 PM

@Quantry

I am inclined to agree that this isn’t the MOST pressing concern. Unless, of course, a glasses-wearer wants to play video games during meetings. Then it would be.

However I was surprised that 83.3% of people interviewed for the research would want to use glass-blur filters in video conferencing platforms.

Glass-blur Filters. Regarding the possible protection of using filters to blur the glass area, 83.3% of the interviewees said they would like to use it;

78.3%, 51.7%, 43.3%, and 11.7% of the 60 interviewees would like to use it when meeting with strangers, colleagues, classes, and family/friends respectively.

They said that Zoom offers a filter for cartoon-like glasses. I checked and can confirm this is true. You can even make your head a cat avatar.

These features are reportedly not in Skype or Google Meet. But perhaps they should be.

Clive Robinson • September 23, 2022 1:46 PM

@ ALL,

The fact they can read big text on glasses with low quality cameras, does not mean that when people switch to high quality cameras taking your glasses off will be enough…

Because they will probably then be able to read the big text off of peoples eyeballs…

But there is something else to remember…

A few years back it was demonstrated by researchers at the UK Cambridge Computer labs that what was on a computer screen facing a wall could be read out using a telescope and photo multiplier.

In essence they read the scanning dot off the wall as it changed it’s luminosity… Thus in effect recreated the serial driver signal to the monitor.

Others there came up with what they called “TEMPEST fonts” designed to have a low EM spectrum output by spreading edge signals across a much wider bandwidth thus reducing the enegy per unit of spectrum bandwidth.

I have a feeling we will be hearing a lot more research in the comming months over “reflected intelligence” and similar in this direction.

Perhaps it’s time to stop using traditional high intensity scanning display technology (which as others have noted is a “health risk”).

Perhaps low energy “Paper White” displays might offer advantages for a while, but at the end of the day…

The current technique only works because of the “red eye effect” where the display the documents are on, form a near 180 degree reflection back to the camera…

People have been using the technique for a couple of centuries at least to “hunt their prey” at dusk through dawn. Look up “lamping” or even “cats eyes”.

foo • September 23, 2022 2:12 PM

My sisters and I all wore glasses as children. We figured out fairly young that you didn’t want a bright light behind you while playing cards because your opponents could see your cards reflected in your glasses.

Same effect I guess with Zoom

John • September 23, 2022 4:22 PM

While the average person likely has little to worry about from such an attack, in the age of social media, any of us could become the target of such an attack. I consider it highly likely that somebody at some point would be greatly embarrassed because some person analyzed a video and found something damning reflected in their eyeglasses. In the worst case, people could even be driven to suicide by internet trolls.

High value targets do need to be wary of such attacks though. I’m sure regular readers of this blog are well aware of the extreme lengths that state-sponsored actors will go to carry out an attack such as this. Every little mishap is a potential opportunity.

JonKnowsNothing • September 23, 2022 5:11 PM

@All

Banks and similar businesses have a monitor-screen filters attached to their systems. It prevents people who are not front facing from seeing what’s on the screen. They have to turn the screen to face you to show you what’s displayed.

So there is some knowledge of screen peeking and some attempt to limit that.

SpaceLifeForm • September 24, 2022 1:27 AM

@ Clive, Ted, ALL

re: “reflected intelligence”

Reminds me of
‘https://www.npr.org/2019/09/05/758038714/can-president-trump-really-tweet-a-highly-classified-satellite-photo-yep-he-can

If you study this pic closely, you can tell that it was pic taken with a cell phone of a monitor, and probably another pic was taken at some point. Maybe 3 times. You can see back-flash artifacts. Note the redaction in the top left corner and how it does not align with the box marking immediately underneath.

Why it was leaked, I’ve no clue. IIRC, there was actually another pic out there that was sharper (taken from a different monitor), but it seems to have been disappeared. Rightly so. IIRC, the redaction in the top left corner was not there when this first leaked, but I do not recall what was there. I would have to think it was a classification marking with TK.

Winter • September 24, 2022 3:41 AM

@john

While the average person likely has little to worry about from such an attack, in the age of social media, any of us could become the target of such an attack.

In any lecture or presentation, a sizeable fraction of the public is reading their email and social media. This fraction is considerably larger during online meetings. Added to that are probably people watching nsfw videos during meetings.

This attack would allow “shoulder” surfing emails, social media, and nsfw content of the participants.

I assume you all can think of situations where colleagues, competitors or enemies who could benefit from such shoulder surfing.

Dancing On Thin Ice • September 24, 2022 7:12 AM

A tip I learned on professional video shoots was to raise eyglasses off the ears to angle the reflections downward away from the camera.

David Leppik • September 24, 2022 11:52 AM

@JonKnowsNothing:

If the user can see what’s on the screen, that’s what is reflected off the glasses.

This is why I’m always careful about who can see me when I punch my passcode into my phone.

Security Sam • September 25, 2022 7:52 AM

When the height of paranoia
Becomes higher than a sequoia
It’s mostly due to the projection
Of one’s own mirror reflection.

Security Sam • September 25, 2022 7:54 AM

When the height of paranoia
Becomes higher than a sequoia
It’s partly due to the projection
Of one’s own mirror reflection.

JonKnowsNothing • September 25, 2022 8:23 AM

@Security Sam

re: Of one’s own mirror reflection.

iirc(badly)

A Buddhist Monk said, “Enlightenment is like looking in a mirror and brushing the dust from the surface.”

Another Buddhist Monk answered, “There is no dust and there is no mirror…”

Peter A. • September 26, 2022 8:37 AM

During most work-related conferences I turn off the camera and most participants do it as well. It’s the presentation and voice that’s relevant, and it saves bandwidth as well. Anyway, I close irrelevant windows/tabs before joining for good measure, specifically when I am going to present something. In some meetings it is customary or recommended to have camera on, but even than it is less than useful. I work with multiple large monitors and full-size keyboard in front of me, so I set the laptop aside and hardly use its screen (mostly to put away annoying “reboot me now” dialogs), and the only camera available is located on the laptop’s frame. So I am pictured at half-profile at best, against a bright background that’s the office window. Good luck getting any reflections off my nose…

The Doctor • October 15, 2022 3:37 PM

Bosses have been using this technique to determine who’s watching porn at work when they should be paying attention to Zoom calls since the Before Times.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.