I’m tired of AI.
I’m tired of hearing about it, of reading about it. I’m tired of otherwise-sensible people falling into foolish behavior around it. I’m tired of the banners and the claims and the manufactured inevitability.
Why yes, I was at BlackHat and DEF CON recently, why do you ask?
In all seriousness: what we are seeing with LLMs right now in security is less revolutionary than you might think. A few months ago I wrote about Thomas Kuhn’s “History of Scientific Revolutions” and the concept of mop-up operations, where science becomes concerned only with filling in the blank spots of the world we think we know. LLMs are excellent attackers in that respect: they are pointed straight at our ever-growing mountains of technical debt, and they can assess and exploit it much, much faster. It’s not creative, and it’s not imaginative. It’s predictive. This point is hard to catch for management folks removed from the actual work. And the answer is not more AI.
The answer - as it almost always is in security - is people.
As mentioned above, I was lucky enough to attend BlackHat and DEF CON again this year. And while companies plastered AI offerings on every flat surface, what actually came out of the woodwork to provide dimensionality and expertise was humanity.
Security professionals need to learn and know AI deeply for all sorts of reasons. Other technologists, hobbyists, or whatever reason you were in Vegas point to the same. That’s undeniable for now. But what creates form and function from start to finish is the people on any given side of our various work equations.
I don’t care about enterprise deployments - so far in practice they’re all prone to hallucination and require expert guidance and handholding for everything but attack - but do I want to hear from the SOC analyst working on a project to tune a model and skills to their environment? Absolutely.
Do I want to talk with the network engineer responsibly building an agentic workflow to measure and monitor baseline activity in their RFC 1918 space? Hell yes.
And on, and on.
It’s the people that matter; I want to quiz the users, I want to listen to the builders. Lessons learned, wins achieved, problems they’re beating their head against that someone may have solved one DEF CON village over.
Hacker Summer Camp delivered in fine style on this front. My first instinct around AI may be curmudgeonly, but like any other technology, hackers are hacking, and that’s still where the magic happens.
Keep on hacking. Then share.
Articles
Gambit Security Threat Intel - AI Across the Intrusion Lifecycle - One of the things I hear most from my blue teamer groups is the lack of technical data and indicators out there regarding AI-augmented attacks. Gambit uses access to attacker infrastructure here to provide a hefty amount of technical data. PDF download, but to their great credit, no email address needed.
Worth reiterating: if you’ve got technical details to share, whatever you publish will get about ten times the traction via community sharing. Defenders are hungry for this. Help us defend.
- Okta Threat Intel - Behind the scenes of a vishing operation - Workflow management breakdown for attack platforms used by at least one SPIDER/SLSH offshoot. Compartmentalized tasking enforced on server-side, and includes project management, I kid you not.
- ReliaQuest - DNS Poisoning Tactics Expand to Hospitality Wi-Fi - Not a surprise, but worth using to communicate the dangers of hotel and event wi-fi to less technical colleagues. Activity aligns to Russian Federation military intelligence.
- FortiGuard - Inside a TrickBot Variant Using DNS Tunneling for C2 - Pretty neat decoding to show what the malformed DNS query and payload are really doing.
- KrebsOnSecurity - Read This Before You Buy That TV Streaming Stick - Krebs has been doing excellent and impactful collaborations on residential proxy or adjacent issues, this go-round with Bitsight. “Smart” televisions and streaming sticks aren’t so much smart as they are malicious and maybe a little cunning.
- DoublePulsar - Adform compromised to serve crypto stealer via supply chain attack - A clear reminder (as if we needed another) that ad-blocking continues to be a critical security activity in both personal and business scenarios.
Research Papers and Reports
- arXiv - Stealing Reasoning Traces from Proprietary LLM APIs - “By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly.” - Clever workaround, though I expect it’ll be patched out soon (relies on common key reuse). I also wonder if the underlying mechanism is tangentially related to previous research on how role confusion contributes to prompt injection.
- arXiv - Signals in the Noise: Open Source Intelligence (OSINT) for AI Loss of Control Detection - Similar to the point above with Gambit, defenders continue to scramble for any possible mitigation or detection to deploy to batten down the hatches. Quite a bit of further exploration and experimentation is necessary, but this is a helpful start.
Tools and Resources
- Jarocki - Pivotglass - “Pivotglass is a local, AI-augmented workspace for cyber-threat investigation.”
.png)
