Have you read about EvilTokens?
Sekoia TDR uncovers EvilTokens, a turnkey Microsoft device code phishing kit sold as Phishing-as-a-Service.
Device code phishing tricks users into entering a code on the legitimate Microsoft login page, granting the attacker’s device access.
Harvested access and refresh tokens can be exchanged for a Primary Refresh Token, enabling 90-day silent access bypassing MFA.
The backend exposes REST endpoints for token weaponisation, cookie generation, Graph reconnaissance and Azure enumeration.
Campaigns deliver phishing pages through PDF, HTML, XLSX, SVG and DOCX lures, hitting organisations across every continent.
Distinctive URL patterns, Cloudflare Workers domains and the X-Antibot-Token header give defenders reliable tracking heuristics.
New widespread EvilTokens kit: device code phishing as-a-service
The clearest example involves a Chinese AI lab. This lab’s systems fetched the secret URLs from more than half of the prompts that we sent to its models between July and October, including roughly two-thirds of the requests marked as “no data collection” or Zero Data Retention. The scraper typically showed up two or three days after the request.
This wasn’t limited to requests sent to the lab directly. Several U.S. hosting providers resell this lab’s models, and prompts sent to those providers triggered the exact same scraper – the prompts were flowing back to the lab. A customer who picked a U.S. provider specifically to keep their data in the U.S. didn’t get what they paid for. Later on, a git client on a different cloud started trying to clone the private repositories referenced in our prompts, but only for prompts that used this lab’s models.
The large U.S. providers have their own issues. One continued to crawl a secret URL more than two months after we used it in a single inference call to their search endpoint. Another runs a scraper that comes back for our URLs days after each request, regardless of which router we used to reach it.
The strangest incident involved a company chat. Twenty-four days after we sent a prompt to one Chinese provider, the secret link from that prompt was posted into a Feishu chat channel (Feishu is known as Lark outside of China). Within two minutes, the link was opened from three desktops and three phones in Beijing and Shanghai. Our disclosure to this provider only included the first eight characters of that link, so whoever shared it was reading from a stored copy of the full prompt, more than three weeks after the request was sent to a U.S.-based provider with the ZDR flag.
I wish they had named these providers. I know they aren’t supposed to, but still.
Name and shame! Bring back the town square and the pillory!
Good to know. Thanks for sharing!
I feel like if you ever put any credentials or PII in any prompt that isn’t staying local, you should just immediately consider it compromised.
Most people just have no idea what they are doing. It’s incredibly common for workers to put sensitive private information into prompts or upload documents, without even thinking about where they might end up and what’s happening with the data. You can not be certain that e.g. a government employee or someone working in healthcare, at an insurance company, etc. isn’t forwarding your private data to some AI service they don’t understand.
Anything going off to someone elses server is not in your control any more.
Lol I run the ai horde which is just Foss and crowdsourced and we’re upfront we cannot guarantee privacy of prompts, but we’re still better than this shite.
Thing about hosting AI Horde is you don’t know what unspeakable prompt your machine will process. Or from where. While I don’t mind this personally (go to town with depraved prompts, it’s not my business), I’m somewhat scared of legal repercussions, getting mixed up in a honeypot, or something like that.
That, and I can’t afford to run my GPU like that during the RAMpocalypse :(
I do agree that it’s better though.
The AI Horde does allow for running in non-NSFW mode, and the NSFW mode itself uses both a blocklist of common keywords and a image recognition based approach to prevent generation of CSAM. IIRC, it is also the basis for blocking out CSAM in lemmy too - there was a short and very ugly phase where federation of illegal stuff made admin lives not fun. I am a daily user of lemmy since the Reddit API exodus, and haven’t yet stumbled across a single instance of CSAM while using accounts on 3 different instances, so it seems to work pretty fine.
I was thinking of the text generation part of the Horde specifically, which doesn’t have such a toggle/filter AFAIK and tends to get used for NSFW in general.
Again, not that I personally mind others’ wild fanfiction and RP.
Shocking but not surprising.
The worst part is that some of the patterns match what you’d expect an AI breakout to be.
The attacker is using the AWS keys to spin up more AI hosting, in theory to spread further






