Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

Post reply on HN
Print view View original

71–80 of 188 posts

Re: AI is breaking two vulnerability cultures

#71

Post by busterarm »

tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

I always understood the business reasons that brought about coordinated vulnerability disclosure & I've been forced to toe this line at employers, but I've always been firmly in the full disclosure camp. I am so ready for this.

Re: AI is breaking two vulnerability cultures

#72

Post by apnorton »

idopmstuff wrote: post #60
pier25 wrote: post #49

Earlier quoted context omitted.

> before it gets better How is it going to get better?

If we assume that there will be an AI that is perfect in terms of ability to find vulnerabilities, cheap to run and widely available to everyone, then anyone can run it on any piece of software before deploying it. All vulnerabilities get found before they can be exploited. One of the big challenges with cybersecurity is that attackers only need to find one exploit, while defenders need to stop everything. When you h…

> If we assume that there will be an AI that is perfect in terms of ability to find vulnerabilities

...so if we assume a halting oracle?

Re: AI is breaking two vulnerability cultures

#73

Post by 0xbadcafebee »

jiggawatts wrote: post #56
pier25 wrote: post #49

Earlier quoted context omitted.

> before it gets better How is it going to get better?

Bulk rewrites of everything into Rust with AI assistance?

Rust is overly complex and difficult, Go is simpler and easier and has the memory protection people are obsessed with

Re: AI is breaking two vulnerability cultures

#74

Post by Worf »

jefftk wrote: post #70
Worf wrote: post #64

> On the other side you have "bugs are bugs" culture. This is especially common in Linux, where the argument is that if the kernel is doing something it shouldn't then someone somewhere may be able to turn it into an attack. Just fix things as quickly as possible, without drawing attention to them. Often people won't notice, with so many changes going past, and there's still time to get machines patched. The 3rd one…

> 90 days is ridiculous, especially for companies It depends on the kind of vulnerability, but sometimes in order to fix a problem, you need to do an enormous amount of software engineering. Which needs to be done to a very high standard, because the expectation is that people will push security patches more or less immediately to production. Of course, this only works if no one else is likely to discover the vulnera…

The company can almost always shut down their service until they fix it. They'll lose money and their customers could also lose money if they depend on the service. That's the price they'll have to pay. Otherwise, they should either work frantically 24/7 to fix the vuln or if they can't, they should accept the fact that they've pushed code without any regard for security and bear the consequences.

Why do we need to put up with excuses? If a company has lots of complicated code that would need enormous amount of time to fix, it's on them. They decided to release this code into the wild.

If I publish the vuln publicly, the users would have the option to stop using the software/service until it's patched. If a customer is using a service without caring about security, it's on them. I want to protect the customers who would monitor the news for such vulns and protect themselves.

Re: AI is breaking two vulnerability cultures

#75

Post by watusername »

nicce wrote: post #55
j2kun wrote: post #27

> Luckily AI can speed up defenders as well as attackers here, allowing embargoes that would previously have been uselessly short. This is an important facet of the problem space: security risks turning into an arms race for who wants to spend more tokens.

One interesting thing is that this makes closed source code even greater asset for the defenders. Attacker cannot spend tokens for it, but defenders can spend tokens for hardening based on source code, while attacker is stuck with blackbox testing.

You would be surprised how adept SOTA models are at reverse engineering with IDA/Ghidra or even plain old objdump. Opus basically knows IDAPython on the back of its hand.

Re: AI is breaking two vulnerability cultures

#76

Post by nicce »

watusername wrote: post #75
nicce wrote: post #55

Earlier quoted context omitted.

One interesting thing is that this makes closed source code even greater asset for the defenders. Attacker cannot spend tokens for it, but defenders can spend tokens for hardening based on source code, while attacker is stuck with blackbox testing.

You would be surprised how adept SOTA models are at reverse engineering with IDA/Ghidra or even plain old objdump. Opus basically knows IDAPython on the back of its hand.

They can be, but the most interesting parts (backend code, deployment confs) are not usually available. Reversing clients can help to understand a bit, but not with equal level.

Re: AI is breaking two vulnerability cultures

#77

Post by chuckadams »

0xbadcafebee wrote: post #69
pier25 wrote: post #49

Earlier quoted context omitted.

> before it gets better How is it going to get better?

1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliabilit…

Basically make software development so legally risky that only multi-billion dollar corporations will ever engage in it.

Re: AI is breaking two vulnerability cultures

#78

Post by awesome_dude »

ragall wrote: post #68
awesome_dude wrote: post #42

Earlier quoted context omitted.

You have moved from "We know" to "We have an educated guess" which is the right way to couch things. However I wanted to also point out that relying only on educated guesses can lead us into a position where we are "papering over the cracks" or "addressing the symptoms", not the "underlying cause" Yes, sometimes that's all that can be done, but, also, sometimes it can be more damaging than the cause itself (thinking…

> You have moved from "We know" to "We have an educated guess" No. You kept blabbering about "science" when most uses of knowledge are not about science. The original topic was also definitely not "science": it was about having a reasonable opinion about whether, empirically, the rate of discovery of vulnerabilities is increasing or not.

Trying to reframe this as 'not science' after being caught on a logical fallacy doesn't change the record. You started with a definitive claim ('We know') to shut down a question. When challenged on the lack of causation, you pivoted to 'educated guesses.'

My point remains: if we misattribute the cause of the rising vulnerability rate (discovery vs. creation), our 'educated guesses' will lead to solutions that address the symptoms while the underlying problem continues to fester. Calling precision 'blabbering' is exactly how we end up with the 'false sense of security' mentioned earlier.

Exhibit A:

ragall 2 hours ago | root | parent | prev | next [–]

> How do you know?

We know because we could see the effects of the average rate of vulnerabilities discovery and exploitation, and it's definitely going up very fast. Until recently, vulnerabilities were relatively hard to find, and finding them was done by a very restricted group of people world-wide, which made them quite valuable. Not any more.

Exhibit B:

ragall 2 hours ago | root | parent | next [–]

Very often you only have limited time for investigation and you have to act now. Action is almost always based on educated guesses. reply

Re: AI is breaking two vulnerability cultures

#79

Post by ReptileMan »

grog454 wrote: post #44
tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> It has been over a decade since any ordinary off-the-shelf closed-source software was meaningfully obscured from serious adversaries. Probably goes without saying but the last line of defense is not deploying your software publicly and instead relying on server-client architectures to do anything. Maybe this will be more common as vulnerabilities are more easily detected and exploited. Of course its not always feas…

>Only the undeployed server code has remained private.

How easy to do you this is for LLM to build decent emulator of the server in question by just observing what you send and what you get as response?

Re: AI is breaking two vulnerability cultures

#80

Post by fragmede »

whattheheckheck wrote: post #63
fragmede wrote: post #54

Earlier quoted context omitted.

Reminder: the Ksplice patent expires October 1, 2028.

What's the implications to that

Means you wouldn't have to reboot to patch for security updates to the Linux kernel. Assuming someone does something with that.
Post reply on HN