Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

Post reply on HN
Print view View original

151–160 of 188 posts

Re: AI is breaking two vulnerability cultures

#151

Post by jefftk »

brabel wrote: post #150
jefftk wrote: post #83

Earlier quoted context omitted.

It's likely varies enormously between projects. Linux remains extremely low in slop, and the vulnerabilities being fixed are quite old, so it's improving. Many vibe coded projects are very sloppy, and are adding a lot of vulnerabilities. Total number of vulnerabilities likely goes up over time weighting all projects equally, but goes down over time weighting by usage.

Is there evidence serious vulnerabilities are the result of vibe coding already? I haven’t seen any so if you have some references, please share.

Security researcher Dor Zvi and his team at the cybersecurity firm he cofounded, RedAccess, analyzed thousands of vibe-coded web applications created using the AI software development tools Lovable, Replit, Base44, and Netlify and found more than 5,000 of them that had virtually no security or authentication of any kind. Many of these web apps allowed anyone who merely finds their web URL to access the apps and their data. Others had only trivial barriers to that access, such as requiring that a visitor sign in with any email address. Around 40 percent of the apps exposed sensitive data, Zvi says, including medical information, financial data, corporate presentations, and strategy documents, as well as detailed logs of customer conversations with chatbots.

https://www.wired.com/story/thousands-of-vibe-coded-apps-exp...

Re: AI is breaking two vulnerability cultures

#152

Post by vanviegen »

deng wrote: post #141
tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! Care to mention these reasons? With "convenience of system administrators", I'm guessing you mean that there's a patch available that sysadmins can install, ideally before the vulnerability is disclosed? What else are sysadmins supposed to d…

Many vulnerabities seem to be in code paths for rarely used features. They can often be disabled.

Re: AI is breaking two vulnerability cultures

#153

Post by shakna »

sedatk wrote: post #112
dataflow wrote: post #102

Earlier quoted context omitted.

Sounds dubious, do you have a citation? The disassembly looks very straightforward for a lot of Windows code.

They're not encoded, but the code blocks are shuffled. That's why disassembly does look straightforward, but it used to thwart BinDiff at the time.

That sounds a lot like US9116712, but I don't think its ever been publicly said that Windows does this.

Re: AI is breaking two vulnerability cultures

#154

Post by cineticdaffodil »

tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

So what materializes now is basically tech debt returns on the "move fast and break things" paradigm?

Re: AI is breaking two vulnerability cultures

#155

Post by cstdr »

sedatk wrote: post #121
dataflow wrote: post #119

Earlier quoted context omitted.

What made you believe this is the case? any examples/links/etc.?

It was a part of our Windows build process when I was at Microsoft. I only assumed that they would keep doing it, but they might have as well dropped the practice.

I don't see how that can be useful when Microsoft publishes debug symbols for almost everything.

Re: AI is breaking two vulnerability cultures

#156

Post by jcgrillo »

pier25 wrote: post #49
Animats wrote: post #46

We have a huge problem. The US is at war. Much of the world is at war at the cyber attack level right now. The US, the EU, most of the Middle East, Israel, Russia... Major services have been attacked and have gone down for days at a time - Ubuntu, Github, Let's Encrypt, Stryker. Entire hospital systems have had to partially shut down. Now, in the middle of this, AI has made attacks much faster to generate. Faster tha…

> before it gets better How is it going to get better?

By holding engineers accountable for the stability and security of their software.

Re: AI is breaking two vulnerability cultures

#157

Post by cush »

rikafurude21 wrote: post #3

This feels more like an old problem getting reframed as an AI problem. people were already diffing kernel commits and figuring out which ones were security fixes long before llms. if a patch lands publicly, the race has basically already started. also not sure shorter embargoes really help. the orgs that can patch in hours are already fine. everyone else still takes days or weeks. if anything, cheaper exploit generat…

> This feels more like an old problem getting reframed as an AI problem.

Sure the mechanics haven't changed, but the scale and risk has. The number of attackers capable of taking advantage of the vulnerability has dramatically increased.

Re: AI is breaking two vulnerability cultures

#158

Post by akerl_ »

riknos314 wrote: post #138
tptacek wrote: post #117

Earlier quoted context omitted.

Right, all I'm saying is that we were asymptotically close many years ago; all that's changed is that nobody can kid themselves about it anymore. The actual policy responses to it, I couldn't say! I've always believed, even when there was a meaningful gap between patching and disclosing, that coordinated disclosure norms were a bad default .

What process or mechanism would you prefer to use instead of coordinated disclosure?

The most common alternative is full disclosure.

Re: AI is breaking two vulnerability cultures

#159

Post by Jach »

deng wrote: post #141
tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! Care to mention these reasons? With "convenience of system administrators", I'm guessing you mean that there's a patch available that sysadmins can install, ideally before the vulnerability is disclosed? What else are sysadmins supposed to d…

The best convenience is that by the time of disclosure, the patch was already merged perhaps months prior and so sysadmins following a routine update schedule would have already updated to a version including the patch and thus have nothing to do. This relies on an assumption that a patch or series of patches aren't equivalent to a disclosure, so that a disclosure can be delayed from the patch, which is basically untenable in modern times.

Re: AI is breaking two vulnerability cultures

#160

Post by tptacek »

deng wrote: post #141
tptacek wrote: post #41

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-sourc…

> based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! Care to mention these reasons? With "convenience of system administrators", I'm guessing you mean that there's a patch available that sysadmins can install, ideally before the vulnerability is disclosed? What else are sysadmins supposed to d…

Choose to take an availability hit rather than risk a breach.
Post reply on HN