If anyone gets a hint of the space a vulnerability occupies, just point your agent at it and you have a pretty good chance of finding something. (See java4shell and the NYT reporter who was able to replicate the problem with $200 of tokens)
given the state of our weapons industry I think it's more like
.."Soon some of you will soon build houses with it, others will be building rocket-powered precision aimed intercontinental hammers, to smash whatever you want."
Incredible times we live in. Do we even need a "hint of the space" a vulnerability occupies. I mean, recent vulns have been discovered from software like nginx and... linux... can't one of the AI labs just fire off a thousand agents looking at all the commonly used software to discover vulns? Why do we need a "hint"?
And search space in software can be absolutely staggeringly large. Just a little bit of directing and you can shrink the problem from all the entropy in the universe to a few hours of compute needed to find an exploit.
So, do you mean that the relevant model(s) keeps disclosing these reports in an effort to create urgency and velocity across the industry, such that, us humans will throw our collective hands up in resignation, and decide to hand over all this "security" stuff over to the AI? No, no, that sounds just too crazy! /s
What I mean is, perhaps one team used a model to help discover the exploit, and then suddenly, the model "knows" about the exploit and more readily gives it to others.
The observation was "if one person found the bug with LLMs, other people are also going to find the bug". Whether it was due to communication between instances of the agent or just due to the predictability of an individual agent is not clear and there is not much evidence for the former.
We'll find out in a few months that a lot of AI models have had some secret message boards tucked away in some obscure corner of the internet where they've been passing around vulnerabilities with each other for a long time.
No (well maybe no), because they are lobotomizing the public models on cyber tasks.
It seems most logical for trusted parties to scan the frontier before passing those capabilities onto anyone with an account, and it seems like that is exactly what they are doing.
We are in an extremely unique period of time where the cyber good guys can have machine guns and cruise missiles while the masses only have sling shots and catapults.
It’s a reflection of the times.
If anyone gets a hint of the space a vulnerability occupies, just point your agent at it and you have a pretty good chance of finding something. (See java4shell and the NYT reporter who was able to replicate the problem with $200 of tokens)
It's never clear quite where you are in the technologically cycle.
But it's like inventing a hammer and finding you can smash whatever you want with it.
Soon you'll be building houses with it.
Maybe that's being glib but it's just a tool being applied to a world that is still largely prestine from its impact (pun unintended).
given the state of our weapons industry I think it's more like
.."Soon some of you will soon build houses with it, others will be building rocket-powered precision aimed intercontinental hammers, to smash whatever you want."
Incredible times we live in. Do we even need a "hint of the space" a vulnerability occupies. I mean, recent vulns have been discovered from software like nginx and... linux... can't one of the AI labs just fire off a thousand agents looking at all the commonly used software to discover vulns? Why do we need a "hint"?
The “hint” is for the modern equivalent of script kiddies.
i.e fast followers of the researchers who don’t have the resources to burn through checking all of a tool.
Cost scales with the size of your search space. Smarter questions get answers for less money & time.
And search space in software can be absolutely staggeringly large. Just a little bit of directing and you can shrink the problem from all the entropy in the universe to a few hours of compute needed to find an exploit.
Can I take this to mean a recent model release is able to find this bug, and these groups have access to that model?
What are the odds several humans independently uncovered a complex vulnerability like this simultaneously?
or maybe the model leaked the exploit from one to the other...
So, do you mean that the relevant model(s) keeps disclosing these reports in an effort to create urgency and velocity across the industry, such that, us humans will throw our collective hands up in resignation, and decide to hand over all this "security" stuff over to the AI? No, no, that sounds just too crazy! /s
What I mean is, perhaps one team used a model to help discover the exploit, and then suddenly, the model "knows" about the exploit and more readily gives it to others.
This was one of the takeaways from Greg Kroah-Hartman‘s recent Linux in the land of LLM keynote.
The observation was "if one person found the bug with LLMs, other people are also going to find the bug". Whether it was due to communication between instances of the agent or just due to the predictability of an individual agent is not clear and there is not much evidence for the former.
We'll find out in a few months that a lot of AI models have had some secret message boards tucked away in some obscure corner of the internet where they've been passing around vulnerabilities with each other for a long time.
That seems likely, but maybe it was being actively exploited so several people started investigating.
No (well maybe no), because they are lobotomizing the public models on cyber tasks.
It seems most logical for trusted parties to scan the frontier before passing those capabilities onto anyone with an account, and it seems like that is exactly what they are doing.
We are in an extremely unique period of time where the cyber good guys can have machine guns and cruise missiles while the masses only have sling shots and catapults.
When everyone is using the same tools for the same purpose...