Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

91–100 of 188 posts

Re: AI is breaking two vulnerability cultures

#91
post #50
post #49

Earlier quoted context omitted.

> before it gets better How is it going to get better?

I'd speculate that at this point Linux etc are probably having vulnerabilities discovered and patched faster than created.

It's not only Linux though and many projects don't have the funding to perpetually use something like Mythos.

Re: AI is breaking two vulnerability cultures

#92

We need automated patch and release cycles. So far we've relied on incredibly slow manual processes to accept reports, investigate, verify, patch, and prepare releases. Releasing a fix often takes months. This is way too slow when attackers can just churn out new exploits in hours. We need to iterate on value chain bottlenecks to lower Mean Time To Patch . We should be able to turn around a bug report to a patched pr…

On the other hand, automated fast rollouts leads to a crowdstrike type situation where you brick all the computers of the world immediately.

Imo we are going to have to rely on more layers of security. Systems that are designed to be secure even in the presence of individual vulnerabilities. This has already been happening for a while on mobile platforms and game consoles. Even physical hardware designed to keep particular secrets /keys even from the kernel.

Re: AI is breaking two vulnerability cultures

#93
post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> BinDiff: you can't patch software without disclosing vulnerabilities

That’s why Microsoft has been obfuscating its binary builds for at least the last two decades so that even the two builds from the same source would produce very different blobs.

Re: AI is breaking two vulnerability cultures

#94
post #76

Earlier quoted context omitted.

You would be surprised how adept SOTA models are at reverse engineering with IDA/Ghidra or even plain old objdump. Opus basically knows IDAPython on the back of its hand.

They can be, but the most interesting parts (backend code, deployment confs) are not usually available. Reversing clients can help to understand a bit, but not with equal level.

On the other hand, any source code leak could be catastrophic

Re: AI is breaking two vulnerability cultures

#95
post #93
post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> BinDiff: you can't patch software without disclosing vulnerabilities That’s why Microsoft has been obfuscating its binary builds for at least the last two decades so that even the two builds from the same source would produce very different blobs.

How are they obfuscated?

Re: AI is breaking two vulnerability cultures

#97

Earlier quoted context omitted.

>Only the undeployed server code has remained private. How easy to do you this is for LLM to build decent emulator of the server in question by just observing what you send and what you get as response?

not sure why downvoted. server emulators will become faster to make. protocol analysis will become faster as well.

Because while you could get something that drives a dumb interface, by moving the work and data to the server it's not available for the emulation software to use.

Re: AI is breaking two vulnerability cultures

#98
post #85
post #33

Earlier quoted context omitted.

You could have a web of trust where Linux-using organizations each spend $x continuously scanning and patching their own dependencies with AI, and sending each other patches and scans.

LLMs aren't capable of doing this, and never will be no matter what Anthropic tries tell you.

That's the same mindset some people had 3 years ago when they said AI wouldn't be capable of software development. Look where we are now.

Re: AI is breaking two vulnerability cultures

#99
Obviously the solution is for Linux to move to a closed-source development model.

Security researchers should report their findings to a committee that includes some big companies (IBM and Oracle seem like trustworthy choices here, but ideally we should find a way to get Microsoft included). Those companies would apply the security patches and distribute binary builds of Linux to their customers. Users fortunate enough to have a business relationship with those companies would be protected immediately. The source would still be published after 90 days for educational purposes and for anyone who doesn't appreciate the security benefits of this approach.

"But even if you could convince people to collaborate like this for the greater good, the GPL makes it legally impossible", you say. Ah, but the GPL only says you have to make the source available for a minimal monetary cost, it doesn't impose a time limit. Traditionally, responding to source code requests with a snail-mailed CD is good enough. No judge in the US is going to rule that a short administrative delay in sending out those CDs - in the name of everyone's security, after all, and 90 days is nothing to the judicial system - violates a nebulous licensing agreement from a different era.

Re: AI is breaking two vulnerability cultures

#100
post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

You’re obviously one of the most knowledgeable people on this topic around here.

What would the best solution be? And where do you believe the industry is headed (which may very well be something other than the best solution) ?

I can’t think about anything other than improving operations, but given the state of the industry, this seems like a pipe dream.

Post reply on HN