[deleted]
Cybersecurity looks like proof of work now
121–130 of 230 posts
Re: Cybersecurity looks like proof of work now
#122Earlier quoted context omitted.
It’s a good thing to keep in mind, but LLM + scaffolding is clearly superior. So if you just use vanilla LLMs you will always be behind. I think the important thing is to avoid over-optimizing. Your scaffold, not avoid building one altogether.
It's wild to me that a paragraph or 7 of plain English that amounts to "be good at things" is enough to make a material difference in the LLM's performance.
They have no values of their own, so you have to direct their attention that way.
Re: Cybersecurity looks like proof of work now
#123I remain skeptical, security is not a notch that you can turn, you can't shove more money or more tokens and make the thing more security.
Not saying security will never be dominated by AI like it happened with chess, with maps, with Go, with language. But just braindead money to security pipeline? Skeptical.
Re: Cybersecurity looks like proof of work now
#124There are never ending ways to make agents better at hacking. Defense is clearly behind. At my startup we are constantly coming up with new defensive measures to put our hacking agent Sable against, and I've determined that you basically need to be air gapped in the future for a chance of survival. A SOC of AI agents can't keep up with 1 AI hacker on a network that is even remotely stealthy. it is a disaster. wrote a…
“Sable began with an initial port scan of 10.10.1.10 and then authenticated to the target.”
That would have started a P2 and woken up a senior IR responder anywhere that I’ve worked. Are you sure you’re running a realistic defender environment?
Re: Cybersecurity looks like proof of work now
#125Am I the only one who thinks this is exactly like it was before AI, when we used small batch hand crafted tokens made by organic engineers to find vulnerabilities?
These mass-produced tokens are just cheaper...
Re: Cybersecurity looks like proof of work now
#126> Cybersecurity looks like proof of work now Imo, cybersecurity looks like formally verified systems now. You can't spend more tokens to find vulnerabilities if there are no vulnerabilities.
I think there's definitely more scope for ruling out vulnerabilities by implementing simpler designs and architectures.
Re: Cybersecurity looks like proof of work now
#127Am I the only one who thinks this is exactly like it was before AI, when we used small batch hand crafted tokens made by organic engineers to find vulnerabilities? These mass-produced tokens are just cheaper...
Cheaper and more fungible. Companies pay lots of money for mediocre security audits. Most attackers aren't very good either. However it only takes one good attacker.
If the attacker and defender are using the same AI model, then (up to some inflection point) whoever spends more finds the most vulnerabilities.
Re: Cybersecurity looks like proof of work now
#128As a result of all this AI "find a zero-day" business, when I boot to windows I open the task manager and order by pid. I kill anything I didn't start or don't recognise. The only process that scared me was windowgrid. It kept finding a way back when I killed all the "start with boot" locations I know. Run, runonce, start up apps, etc. Surely it's not in autoexec.bat :)
> when I boot to windows
Re: Cybersecurity looks like proof of work now
#129There's still the question of access to the codebase. By all accounts, the best LLM cyber scanning approaches are really primitive - it's just a bash script that goes through every single file in the codebase and, for each one and runs a "find the vulns here" prompt. The attacker usually has even less access than this - in the beginning, they have network tools, an undocumented API, and maybe some binaries. You can d…
>By all accounts, the best LLM cyber scanning approaches are really primitive
It seems like that is perhaps not the case anymore with the Mythos model?
Re: Cybersecurity looks like proof of work now
#130If you have a limited budget of tokens as a defender, maybe the best thing to spend them on is not red teaming, but formalizing proofs of your code's security. Then the number of tokens required roughly scales with the amount and complexity of your code, instead of scaling with the number of tokens an attacker is willing to spend. (It's true that formalization can still have bugs in the definition of "secure" and doe…
Is it possible to prove security properties about a web application?