Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

181–188 of 188 posts

Re: AI is breaking two vulnerability cultures

#181

Earlier quoted context omitted.

What process or mechanism would you prefer to use instead of coordinated disclosure?

I guess people could download (but not install) encrypted patches with an announced key release date+time, so that by the moment it is disclosed essentially everyone is applying the patch.

That's still coordinated, but by the publicizing of the key

Re: AI is breaking two vulnerability cultures

#182
post #36

The old saying of Tony Hoare about no obvious bugs vs obviously no bugs holds in the age of LLMs more than ever

> There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies." — Tony Hoare for those (like me) who hadn't seen it before.

That’s really clever and quite funny!

Re: AI is breaking two vulnerability cultures

#183

solution is simple: commits get shared privately. distros can build on these private patches. after some point those can be made public (ie commited at upstream)

Source needs to be closed too practically speaking.

But I bet AI is pretty good at analyzing binary too.

Fix your bugs!

Re: AI is breaking two vulnerability cultures

#184
post #66
post #47

I must admit I'm rather enjoying this particular form of shit show, mostly because it was a predication I made in 2023 in the early days of LLMs. It wasn't really a problem related to LLMs but a glaring hole in the thinking of current computing which is the "frustratingly over-connected" and "over-trust" approach to everything. After reading Liu Cixin's "three body problem" and noting the Dark Forest, I applied that…

If you're into gaming, Cyberpunk 2077 essentially plays in a heavily technologized world, where all compute infrastructure is infested with rogue AI that replicates itself to any technology it can physically get in touch with. The only recourse is a new web, built from first principles, protected by (probably) benevolent AI systems. Every device, every server, is partially occupied by AIs doing their thing on it, vir…

Have played that end to end. It's a dystopian hell hole.

Re: AI is breaking two vulnerability cultures

#185
post #93
post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> BinDiff: you can't patch software without disclosing vulnerabilities That’s why Microsoft has been obfuscating its binary builds for at least the last two decades so that even the two builds from the same source would produce very different blobs.

All the while, Linux is going towards reproducible builds (Debian just announced it as a policy). This is of course the only sane way for FOSS, and, I believe, the only sane long term approach in any case. Security by obscurity, while not worthless, is just a thin mitigation layer. By the way, build-time randomization is ineffective in light of AI analysis---it needs to be per-binary-run, in the style of KASLR.

Re: AI is breaking two vulnerability cultures

#186
post #49
post #46

We have a huge problem. The US is at war. Much of the world is at war at the cyber attack level right now. The US, the EU, most of the Middle East, Israel, Russia... Major services have been attacked and have gone down for days at a time - Ubuntu, Github, Let's Encrypt, Stryker. Entire hospital systems have had to partially shut down. Now, in the middle of this, AI has made attacks much faster to generate. Faster tha…

> before it gets better How is it going to get better?

Building everything better with more secure tech instead of just doing what we were doing before.

So, bsd pledge-likes, capabilities, design-by-contract, linear types, effect systems, ZKPs, provably correct systems, etc...

Re: AI is breaking two vulnerability cultures

#187
post #84
post #74

Earlier quoted context omitted.

The company can almost always shut down their service until they fix it. They'll lose money and their customers could also lose money if they depend on the service. That's the price they'll have to pay. Otherwise, they should either work frantically 24/7 to fix the vuln or if they can't, they should accept the fact that they've pushed code without any regard for security and bear the consequences. Why do we need to p…

How would you apply this logic to something like https://meltdownattack.com ? The vulnerability was in hardware, discovered by companies that make user level software, and mitigated by changes to OS kernels.

Sorry for the late reply. While that's a good example of a vuln that's not "owned" by the affected party and mitigation would be hard to create and distribute, IMO we should still make it public relatively quickly because we can't know whether it has already been discovered by someone else. The public as a whole will likely create countermeasures like patches or workarounds more quickly than a small subset comprising of OS developers and CPU vendors. Personally, I might heavily restrict using JS on random sites or downloading random binaries or scripts (even if they're virtualized), virtualize whatever I can (it seems the 2 previous actions are contradictory, but virtualization could help so why not), separate some data and processes on different physical machines or use a different CPU architecture for some things.

Re: AI is breaking two vulnerability cultures

#188
post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

I believe this premise that the cost of identification of vulnerabilities via diffs is going down over time begs the question "what do our processes need to look like if simply making the patch public is the disclosure?" Current coordinated disclosure practices have a dependency on patching and disclosure being separate, but the gap between them seems to be asymptomatically approaching zero.

[flagged]
Post reply on HN