Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cybersecurity looks like proof of work now

dbreunig.com

Post reply on HN
Print view View original

71–80 of 230 posts

Re: Cybersecurity looks like proof of work now

#71

Post by SyneRyder »

snowwrestler wrote: post #4

It looks like proof of work because: > Worryingly, none of the models given a 100M budget showed signs of diminishing returns. “Models continue making progress with increased token budgets across the token budgets tested,” AISI notes. So, the author infers a durable direct correlation between token spend and attack success. Thus you will need to spend more tokens than your attackers to find your vulnerabilities first…

Worth pointing out that as impressive as the 32-step network takeover is, Mythos wasn't able to achieve it on every attempt, and the network itself did not have the usual defence systems.

I wouldn't use those as excuses to dismiss AI though. Even if this model doesn't break your defences, give it 3 months and see where the next model lands.

Re: Cybersecurity looks like proof of work now

#73

Post by mikestaas »

stackghost wrote: post #58
jeffmcjunkin wrote: post #41

Earlier quoted context omitted.

Can confirm. Matching decompilation in particular (where you match the compiler along with your guess at source, compile, then compare assembly, repeating if it doesn't match) is very token-intensive, but it's now very viable: https://news.ycombinator.com/item?id=46080498 Of course LLMs see a lot more source-assembly pairs than even skilled reverse engineers, so this makes sense. Any area where you can get unlimited…

My own experience has been that "ghidra -> ask LLM to reason about ghidra decompilation" is very effective on all but the most highly obfuscated binaries. Burning tokens by asking the LLM to compile, disassemble, compare assembly, recompile, repeat seems very wasteful and inefficient to me.

LaurieWired did a good episode about that kind of thing https://www.youtube.com/watch?v=u2vQapLAW88

Re: Cybersecurity looks like proof of work now

#74

Post by samuelknight »

adriancooney wrote: post #72

Does this mean all code written before Mythos is a liability?

All code is a liability in general. All code written before LLMs and during the current in-between years are vulnerable to the next frontier model. Eventually we will settle into a new paradigm that correctly addresses the new balance of effort.

Re: Cybersecurity looks like proof of work now

#75

Post by tptacek »

j2kun wrote: post #68
tptacek wrote: post #25

Earlier quoted context omitted.

I would be interested in which notable security researchers you can find to take the other side of this argument. I don't know anything about the "AI Security Institute", but they're saying something broadly mirrored by security researchers. From what I can see, the "debate" in the actual practitioner community is whether frontier models are merely as big a deal as fuzzing was, or something signficantly bigger. Fuzzi…

It's less that I think they would take the other side of the argument, than that they would lend some credence to the content of the analysis. For example, I would not particularly trust a bunch of AI researchers to come up with a representative set of CTF tasks, which seems to be the basis of this analysis.

Yeah, you might be right about this particular analysis! The sense I have from talking to people at the labs is that they're really just picking deliberately diverse and high-profile targets to see what the models are capable of.

Re: Cybersecurity looks like proof of work now

#76

Post by gerdesj »

chromacity wrote: post #56
Muromec wrote: post #32

Earlier quoted context omitted.

Commercial infosec is deleting firefox from develop machines, because it's not secure and explaining to muggles why they shouldn't commit secret material to the code repository. That and blocking my ssh access to home router of course.

I mean, often, yep. The real reason why they are unhappy with you having an unsupported browser is simply that it's much harder to reason about or enforce policies across bespoke environments. And in an enterprise of a sufficient scale, the probability that one of your employees is making a mistake today is basically 1. Someone is installing an infostealer browser extension, someone is typing in their password on a p…

"The real reason why they are unhappy with you having an unsupported browser"

I tend to encourage Firefox over Cr flavoured browsers because FF (for me) are the absolute last to dive in with fads and will boneheadedly argue against useful stuff until the cows come home ... Web Serial springs to mind (which should finally be rocking up real soon now).

Oh and they are not sponsored by Google errm ... 8)

I'm old enough to remember having to use telnet to access the www (when it finally rocked up and looked rather like Gopher and WAIS) (via a X.25 PAD) and I have seen the word "unsupported" bandied around way too often since to basically mean "walled garden".

I think that when you end up using the term "unsupported browser" you have lost any possible argument based on reason or common decency.

Re: Cybersecurity looks like proof of work now

#77

Post by samuelknight »

I don't know about Mythos but the chart understates the capability of the current frontier models. GPT and Claude models available today are capable of Web app exploits, C2, and persistence in well under 10M tokens if you build a good harness.

The benchmark might be a good apples-to-apples comparison but it is not showing capability in an absolute sense.

Re: Cybersecurity looks like proof of work now

#78

Post by pixl97 »

linkregister wrote: post #49
btown wrote: post #29

Earlier quoted context omitted.

The problem, though, is that this turns "one of our developers was hit by a supply chain attack that never hit prod, we wiped their computer and rotated keys, and it's not like we're a big target for the attacker to make much use of anything they exfiltrated..." into "now our entire source code has been exfiltrated and, even with rudimentary line-by-line scanning, will be automatically audited for privilege escalatio…

This is a great example of vulnerability chains that can be broken by vulnerability scanning by even cheaper open source models. The outcome of a developer getting pwned doesn't have to lead to total catastrophe. Having trivial privilege escalations closed off means an attacker will need to be noisy and set off commodity alerting. The will of the company to implement fixes for the 100 Github dependabot alerts on thei…

I disagree that it's extra time for security, it's the time we should have been spending in the first place.

Re: Cybersecurity looks like proof of work now

#79

Post by danieltk76 »

There are never ending ways to make agents better at hacking. Defense is clearly behind. At my startup we are constantly coming up with new defensive measures to put our hacking agent Sable against, and I've determined that you basically need to be air gapped in the future for a chance of survival. A SOC of AI agents can't keep up with 1 AI hacker on a network that is even remotely stealthy. it is a disaster. wrote an article about it: https://blog.vulnetic.ai/evading-an-ai-soc-with-sable-from-v...
Post reply on HN