Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cybersecurity looks like proof of work now

dbreunig.com

Post reply on HN
Print view View original

181–190 of 230 posts

Re: Cybersecurity looks like proof of work now

#181

Post by fhd2 »

antirez wrote: post #173

Why this is the wrong analogy: finding hash collisions, while exponentially harder with N, is guaranteed to find, with enough work, some S so that H(S) satisfies N, so an asymmetry of resources used will have the side with more work eventually winning. But bugs are different: 1. different LLMs executions take different branches, but eventually the branches possible based on the code possible states are saturated. 2.…

Agreed, it is different in terms of there being no guarantee that a specific piece of software even has an exploit. If you don't want to break into a specific piece of software, or even a specific system, I would argue that the law of averages applies: If you just invest enough, you'll likely find _something_ worth exploiting.

In other terms, I feel the argument from TFA generally checks out, just on a different level than "more GPU wins". It's one up: "More money wins". That's based on the premise that more capable models will be more expensive, and using more of it will increase the likelihood of finding an exploit, as well as the total cost. What these model providers pay for GPUs vs R&D, or what their profit margin is, I'd consider less central.

But then again, AI didn't change this, if you have more money you can find more exploits: Whether a model looks for them or a human.

Re: Cybersecurity looks like proof of work now

#182

Post by mapontosevenths »

janalsncm wrote: post #28
mapontosevenths wrote: post #21

Earlier quoted context omitted.

>in principle, cybersecurity is advantage defender I disagree. The defender must be right every single time. The attacker only has to get lucky and thanks to scale they can do that every day all day in most large organizations.

My understanding of defense in depth is that it is a hedge against this. By using multiple uncorrelated layers (e.g. the security guard shouldn’t get sleepier when the bank vault is unlocked) you are transforming a problem of “the defender has to get it right every time” into “the attacker has to get through each of the layers at the same time”.

It is a hedge, that said it only reduces the probability of an event and does not eliminate it.

To use your example, if the odds of the guard being asleep and the vault being unlocked are both 1% we have a 0.0001 chance on any given day. Phew, we're safe...

Except that Google says there are 68,632 bank branch locations in the US alone. That means it will happen roughly 7 times on any given day someplace in America!

Now apply that to the scale of the internet. The attackers can rattle the locks in every single bank in an afternoon for almost zero cost.

The poorly defended ones have something close to 100% odds of being breached, and the well defended ones how low odds on any given day, but over a long enough timeline it becomes inevitable.

To again use your bank example. if we only have one bank, but keep those odds it means that over about 191 years the event will happen 7 times. Or to restate that number, it is like to happen at least once every 27 years. You'll have about 25% odds of it happening in any 7 year span.

For any individual target, it becomes unlikely, but also still inevitable.

From an attackers perspective this means the game is rigged in their favor. They have many billions of potential targets, and the cost of an attack is close to zero.

From a defenders perspective it means realizing that even with defense in depth the breach is still going to happen eventually and that the bigger the company is the more likely it is.

Cyber is about mitigating risk, not eliminating it.

Re: Cybersecurity looks like proof of work now

#183

Post by Kinrany »

anitil wrote: post #53
somesortofthing wrote: post #7

There's still the question of access to the codebase. By all accounts, the best LLM cyber scanning approaches are really primitive - it's just a bash script that goes through every single file in the codebase and, for each one and runs a "find the vulns here" prompt. The attacker usually has even less access than this - in the beginning, they have network tools, an undocumented API, and maybe some binaries. You can d…

On that latest episode of 'Security Cryptography Whatever' [0] they mention that the time spent on improving the harness (at the moment) end up being outperformed by the strategy of "wait for the next model". I doubt that will continue, but it broke my intuition about how to improve them [0] https://securitycryptographywhatever.com/2026/03/25/ai-bug-f...

That only applies to workarounds for current limitations, no? Some things a harness can do will apply in the same way to future models.

Re: Cybersecurity looks like proof of work now

#184

Post by wg0 »

j2kun wrote: post #20

The article heavily quotes the "AI Security Institute" as a third-party analysis. It was the first I heard of them, so I looked up their about page, and it appears to be primarily people from the AI industry (former Deepmind/OpenAI staff, etc.), with no folks from the security industry mentioned. So while the security landscape is clearly evolving (cf. also Big Sleep and Project Zero), the conclusion of "to harden a…

If true, that's naked, shameless and brutal capitalism.

Seems much like those secretly tobacco industry funded reports about tobacco being safe and such.

Re: Cybersecurity looks like proof of work now

#185

Post by tossandthrow »

somesortofthing wrote: post #7

There's still the question of access to the codebase. By all accounts, the best LLM cyber scanning approaches are really primitive - it's just a bash script that goes through every single file in the codebase and, for each one and runs a "find the vulns here" prompt. The attacker usually has even less access than this - in the beginning, they have network tools, an undocumented API, and maybe some binaries. You can d…

> it's just a bash script that goes through every single file in the codebase and, for each one and runs a "find the vulns here" prompt.

This really is not the case.

You have freedom of methodology.

You can also ask it to enumerate various risks and find proof of existence for each of them.

Certainly our LLM audits are not just a prompt per file - so I have a hard time believing that best in class tools would do this.

Re: Cybersecurity looks like proof of work now

#186

Post by mapontosevenths »

traderj0e wrote: post #22
mapontosevenths wrote: post #21

Earlier quoted context omitted.

>in principle, cybersecurity is advantage defender I disagree. The defender must be right every single time. The attacker only has to get lucky and thanks to scale they can do that every day all day in most large organizations.

Well, the attacker has something to lose too. It's not like the defender has to be perfect or else attacks will just happen, it takes time/money to invest in attacking.

The cost to your average ransomware crew can be rounded down to zero, because it's pretty darn close. They use automated tools running on other peoples computers and utilizing other peoples connectivity. The tools themselves for most RaaS (ransomware as a service) affiliates are also close to zero cost, as they pay the operator a percentage of profits.

The time is a cost, but at scale any individual target is a pretty minor investment since it's 90%+ automated. Also, these aren't folks that are otherwise highly employable. The opportunity cost to them is also usually very low.

The last attacker I got into a conversation with was interesting. Turns out, he was a 16 year old from Atlanta GA using a toolkit as an affiliate. He claimed he made ~100k/year and used the money on cars and girls. I felt like he was inflating that number to brag. His alternative probably would have been McDonalds, and as a minor if he got caught it would've been probation most likely. I told him to come to the blue team, we pay better.

Re: Cybersecurity looks like proof of work now

#187

Post by alexwebb2 »

jzelinskie wrote: post #19

Security has always been a game of just how much money your adversary is willing to commit. The conclusions drawn in lots of these articles are just already well understood systems design concepts, but for some reason people are acting like they are novel or that LLMs have changed anything besides the price. For example from this article: > Karpathy: Classical software engineering would have you believe that dependen…

> "security through obscurity" has been a well understand fallacy in open source circles for decades

The times, as they say, are a-changin’.

Open software is not inherently more secure than closed software, and never has been.

Its relative security value was always derived from circumstantial factors, one of the most important of which was the combination of incentive and ability and willingness of others in the community to spend their time and attention finding and fixing bugs and potential exploits.

Now, that’s been the case for so long that we all implicitly take it for granted, and conclude that open software is generally more secure than closed, and that security through obscurity falls short in comparison.

But this may very well fundamentally change when the cost of navigating the search space of potential exploits, for both the attacker and the defender, is dramatically reduced along the axes of time and attention, and increased along the axis of monetary investment.

It then becomes a game of which side is more willing to pool monetary resources into OSS security analysis – the attackers or the defenders – and I wouldn’t feel comfortable betting on the defenders in that case.

Re: Cybersecurity looks like proof of work now

#188

Post by eru »

lelanthran wrote: post #164
eru wrote: post #106

Earlier quoted context omitted.

> There's a massive cost asymmetry between the "hardening" phase for the defender and the "discovering exploits" phase for the attacker. Well, you need to harden everything, the attacker only needs to find one or at most a handful of exploits.

> Well, you need to harden everything, the attacker only needs to find one or at most a handful of exploits. Yeah, but it's not like the attacker knows where to look without checking everything, it it? If you harden and fix 90% of vulns, the attacker may give up when their attempts reach 80% of vulns. It's the same as it has ever been; you don't need to outrun the bear, you only need to outrun the other runners.

Compare and contrast https://en.wikipedia.org/wiki/Kerckhoffs%27s_principle

Re: Cybersecurity looks like proof of work now

#189

Post by lelanthran »

eru wrote: post #188
lelanthran wrote: post #164

Earlier quoted context omitted.

> Well, you need to harden everything, the attacker only needs to find one or at most a handful of exploits. Yeah, but it's not like the attacker knows where to look without checking everything, it it? If you harden and fix 90% of vulns, the attacker may give up when their attempts reach 80% of vulns. It's the same as it has ever been; you don't need to outrun the bear, you only need to outrun the other runners.

Compare and contrast https://en.wikipedia.org/wiki/Kerckhoffs%27s_principle

My point is that the cost for the attacker is higher than the cost for the defender, if the attacker has to spend tokens probing for vulnerabilities against a system which has little know about it, while the defender spends tokens on a system they have the full source to.

That is not at all relevant to "security via obscurity" or similar arguments: having the source in the open may (eventually) be more secure, but it lowers the token-spend for the attacker.

Re: Cybersecurity looks like proof of work now

#190

Post by bitexploder »

anitil wrote: post #138
bitexploder wrote: post #110

Earlier quoted context omitted.

And if you have the better harness and the next model?

I would _hope_ that the double combo would be better, but honestly I have no idea

I do. It is better. I have done a lot of vuln research. I can get way better than one shot level results out of “inferior” models.
Post reply on HN