Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

Post reply on HN
Print view View original

111–120 of 188 posts

Re: AI is breaking two vulnerability cultures

#111

Post by skinfaxi »

raincole wrote: post #107
dmurray wrote: post #99

Obviously the solution is for Linux to move to a closed-source development model. Security researchers should report their findings to a committee that includes some big companies (IBM and Oracle seem like trustworthy choices here, but ideally we should find a way to get Microsoft included). Those companies would apply the security patches and distribute binary builds of Linux to their customers. Users fortunate enou…

I like how after so many years, people finally start recognizing that obscurity is a part of security. Not the whole security, obviously, but a part of it.

The comment you replied to read like satire to me.

Re: AI is breaking two vulnerability cultures

#112

Post by sedatk »

dataflow wrote: post #102
sedatk wrote: post #93

Earlier quoted context omitted.

> BinDiff: you can't patch software without disclosing vulnerabilities That’s why Microsoft has been obfuscating its binary builds for at least the last two decades so that even the two builds from the same source would produce very different blobs.

Sounds dubious, do you have a citation? The disassembly looks very straightforward for a lot of Windows code.

They're not encoded, but the code blocks are shuffled. That's why disassembly does look straightforward, but it used to thwart BinDiff at the time.

Re: AI is breaking two vulnerability cultures

#113

Post by sedatk »

wglb wrote: post #95
sedatk wrote: post #93

Earlier quoted context omitted.

> BinDiff: you can't patch software without disclosing vulnerabilities That’s why Microsoft has been obfuscating its binary builds for at least the last two decades so that even the two builds from the same source would produce very different blobs.

How are they obfuscated?

See my sibling comment.

Re: AI is breaking two vulnerability cultures

#114

Post by dakolli »

fragmede wrote: post #88
dakolli wrote: post #85

Earlier quoted context omitted.

LLMs aren't capable of doing this, and never will be no matter what Anthropic tries tell you.

Mozilla seems to think it can. https://blog.mozilla.org/en/privacy-security/ai-security-zer...

Ahh yes, I'm sure agents did this all autonomously without any human in the loop what so ever. They are useless without experts to handle them.

Re: AI is breaking two vulnerability cultures

#115

Post by [deleted] »

awesome_dude wrote: post #78
ragall wrote: post #68

Earlier quoted context omitted.

> You have moved from "We know" to "We have an educated guess" No. You kept blabbering about "science" when most uses of knowledge are not about science. The original topic was also definitely not "science": it was about having a reasonable opinion about whether, empirically, the rate of discovery of vulnerabilities is increasing or not.

Trying to reframe this as 'not science' after being caught on a logical fallacy doesn't change the record. You started with a definitive claim ('We know') to shut down a question. When challenged on the lack of causation, you pivoted to 'educated guesses.' My point remains: if we misattribute the cause of the rising vulnerability rate (discovery vs. creation), our 'educated guesses' will lead to solutions that addres…

[deleted]

Re: AI is breaking two vulnerability cultures

#116

Post by dakolli »

a_vanderbilt wrote: post #98
dakolli wrote: post #85

Earlier quoted context omitted.

LLMs aren't capable of doing this, and never will be no matter what Anthropic tries tell you.

That's the same mindset some people had 3 years ago when they said AI wouldn't be capable of software development. Look where we are now.

I have unlimited access to every single frontier model, I've tested all of them, they are not good at writing software.

They are basically slot machines, sometimes you win a little bit and sometimes you win a lot but usually you just burn a ton of time and money sitting and staring at a screen (and frying your brain).

Re: AI is breaking two vulnerability cultures

#117

Post by tptacek »

riknos314 wrote: post #105
tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

I believe this premise that the cost of identification of vulnerabilities via diffs is going down over time begs the question "what do our processes need to look like if simply making the patch public is the disclosure?" Current coordinated disclosure practices have a dependency on patching and disclosure being separate, but the gap between them seems to be asymptomatically approaching zero.

Right, all I'm saying is that we were asymptotically close many years ago; all that's changed is that nobody can kid themselves about it anymore.

The actual policy responses to it, I couldn't say! I've always believed, even when there was a meaningful gap between patching and disclosing, that coordinated disclosure norms were a bad default.

Re: AI is breaking two vulnerability cultures

#118

Post by dnnddidiej »

chuckadams wrote: post #77
0xbadcafebee wrote: post #69

Earlier quoted context omitted.

1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliabilit…

Basically make software development so legally risky that only multi-billion dollar corporations will ever engage in it.

We could get somewhere where clouds can provide a framework of secure primitives that act as a framework.

E.g. you build an app, it stores data via api etc. etc. You can test in sandbox. The cloud deploys for customer who paid you via that cloud and you work at arms length. You may not even know their name. You just get the pro subscription fees.

The idea bubbling in my head would be an app store for cloud products. But with competition i.e. you use Railway or Heroku or AWS for the best deal.

Be gentle this is an idea in my head I am sure it can be torn down by a retort at this stage. But this exists in forms and I think it will emerge. It is inversion of control at the entire app level.

This is similar to buying a hammer. If you make hammers you sell them to a store, the store knows the customer and only the customer can see the nails.

Re: AI is breaking two vulnerability cultures

#119

Post by dataflow »

sedatk wrote: post #112
dataflow wrote: post #102

Earlier quoted context omitted.

Sounds dubious, do you have a citation? The disassembly looks very straightforward for a lot of Windows code.

They're not encoded, but the code blocks are shuffled. That's why disassembly does look straightforward, but it used to thwart BinDiff at the time.

What made you believe this is the case? any examples/links/etc.?

Re: AI is breaking two vulnerability cultures

#120

Post by mlac »

freeqaz wrote: post #86

This is exactly what happened with Log4Shell. Day -X + 1: Engineer at Alibaba finds the vuln and tells Apache. Patch is pushed to git while new release is coordinated. Day -X: A black hat sees commits fixing the bug. Attacks start happening. Day 0: Memes start circulating in Minecraft communities of people crashing servers. Some logs are shared on Twitter, especially in China, of people getting pwned. Day 0 + ~4 hour…

I’m with you until that last sentence, which I’ve been thinking about as “… until AI code testing, vulnerability scanning, and developer support tools help to limit the number of 0-days and vulnerabilities making it into production”.

So prevention will be more important than ai-assisted rapid containment or patching, though both of those capabilities will be necessary as part of defense in depth.

And some sort of AI-enabled security analysis across the organization’s architecture that is done as part of testing ahead of new software entering production to ID potential vulnerabilities caused by configuration changes or upgrades that modify how systems interact with each other.

I’ve been trying to guess the timeframe for seeing improved secure development, but I’m hoping it’s a bit closer to 6 months - 1 year given the speed of AI adoption and AI progression. May be closer to 3 years as you stated.

In the meantime, is there more to be done than this (not in order)?

- Patch COTS software

- re-evaluate the scoring for previous vulnerabilities

- set up up containment measures capabilities for systems that can’t be patched / high risk vendors

- use frontier model vuln scanning and patching for home grown systems that may have more 0-days than COTS depending on the organization’s capability

- limit the number of vendors / simplifying the tech stack.

I’d be happy to hear how others are thinking about this.

Post reply on HN