Schneier on Security

echo • May 9, 2024 1:20 PM

To improve their social engineering tricks. LLMs prove to be particularly well-suited to the domain of social engineering, where they are able to offer a variety of capabilities. Criminals use such technology for crafting scam scripts and scaling up production on phishing campaigns. Benefits include the ability to convey key elements such as a sense of urgency and the ability to translate text in different languages. While seemingly simple, the latter has proven to be one of the most disruptive features for the criminal world, opening new markets that were previously inaccessible to some criminal groups because of language barriers.

And:

To circumvent this, criminals are now offering to take a stolen ID and create a deepfake image to convince the system of a customer’s legitimacy.

Annoying but limited. The current level of technology is convincing for low bandwidth automated and low task orientated work. I’m sure much worse is coming one day but we’re not there yet.

And:

Problems arise for criminals when a deepfake attack targets somebody close to the subject being impersonated, as deepfake videos have not yet reached a level where they can fool people with an intimate knowledge of the impersonated subject. As a consequence, criminals looking for more targeted attacks toward individuals — like in the case of virtual kidnapping scams — prefer audio deepfakes instead. These are more affordable to create, require less data from the subject, and generate more convincing results. Normally, a few seconds of the subject’s voice can suffice, and this type of audio is often publicly available on social media.

I can be chatty or have behaviours which look like nothing on the surface but pull more information out of another person. Discussion tilt, tone, emotional microexpressions can mean something as can none verbal equivalents, timing, and responses. And also what they don’t say and don’t do. I also have a few tricks like being busy or kicking the can down the road, or putting on an act of agreeing with them while I’m buying time and they’re giving me more information for free, or just waiting to see what happens with other people as part of that feedback/evaluation loop. Sometimes I’m just me being me. It’s hard to say, really.

I never like urgent. The more urgent someone is the less I like it. What are they really after, and how are they lying to me, and what else is happening I don’t know about?

I know from experience and feedback from other sources I made the correct decision when it mattered. I didn’t get it right every single time but you pick up a sense about it. It’s none linear. You can only get that from experience. My toleration for something “off” is low.

Let’s see how far an “AI” gets with an in-person meeting.

Welcome to my web said the spider to the fly.

For anything else I would call the cops.

echo • May 9, 2024 6:27 PM

I question whether cloud based AI providers have the ability or motivation to prevent or even discourage fraudulent behaviors.

I wouldn’t be surprised if the do know and don’t care. See also tobacco, climate change, Brexit, social media etcetera. AI needs regulating like a biohazard. That or people need to discover physical retail banks again.

Madame doesn’t use online banking. If I wanted to I would need to visit my bank so it can be activated. I am in no hurry.