Live data from Hacker News

AI is breaking two vulnerability cultures

jefftk.com

121–130 of 188 posts

Re: AI is breaking two vulnerability cultures

#121
post #112

Earlier quoted context omitted.

They're not encoded, but the code blocks are shuffled. That's why disassembly does look straightforward, but it used to thwart BinDiff at the time.

What made you believe this is the case? any examples/links/etc.?

It was a part of our Windows build process when I was at Microsoft. I only assumed that they would keep doing it, but they might have as well dropped the practice.

Re: AI is breaking two vulnerability cultures

#122
post #97

Earlier quoted context omitted.

not sure why downvoted. server emulators will become faster to make. protocol analysis will become faster as well.

Because while you could get something that drives a dumb interface, by moving the work and data to the server it's not available for the emulation software to use.

Except that emulating what is seen is surprisingly useful to find attack vectors. As a single deeper datapoint, one can look at more than just baseline behavior and delve into timing details to further refine implementation guesses.

Re: AI is breaking two vulnerability cultures

#123
post #97

Earlier quoted context omitted.

not sure why downvoted. server emulators will become faster to make. protocol analysis will become faster as well.

Because while you could get something that drives a dumb interface, by moving the work and data to the server it's not available for the emulation software to use.

If the contract is well defined, the LLM can infer what it's purpose is, implementation, possibly even your secret sauce. There is no software moat anymore.

Re: AI is breaking two vulnerability cultures

#124

Earlier quoted context omitted.

1) Make it a law that companies have to vet their code for security holes before release, 2) Make it a law that companies have to apply operational security best practice on their software products/services, 3) Industry standard automation for improvements to patch lifecycle management, 4) Auditing for critical businesses and industries to ensure safety (both as a national security thing and general safety/reliabilit…

Basically make software development so legally risky that only multi-billion dollar corporations will ever engage in it.

Legal risk is what insurance is for. You get ensured for a small fee and you go about your job. That's how the non-software world operates anyway

Re: AI is breaking two vulnerability cultures

#125
post #120
post #86

This is exactly what happened with Log4Shell. Day -X + 1: Engineer at Alibaba finds the vuln and tells Apache. Patch is pushed to git while new release is coordinated. Day -X: A black hat sees commits fixing the bug. Attacks start happening. Day 0: Memes start circulating in Minecraft communities of people crashing servers. Some logs are shared on Twitter, especially in China, of people getting pwned. Day 0 + ~4 hour…

I’m with you until that last sentence, which I’ve been thinking about as “… until AI code testing, vulnerability scanning, and developer support tools help to limit the number of 0-days and vulnerabilities making it into production”. So prevention will be more important than ai-assisted rapid containment or patching, though both of those capabilities will be necessary as part of defense in depth. And some sort of AI-…

we simply can't absolve ourselves of responsibility in input and expect a hardened output. It's ABSOLUTELY up to the engineers to have test harnesses and scenarios for testing, vulnerability scanning, etc. Just because we can move faster via prompts doesn't mean we neglect the SDLC.

I think there's opportunity to reinvent the pipeline with AI powered tools to assist but the onus is still on the person to ensure they are deploying something that has been tested.

Re: AI is breaking two vulnerability cultures

#126

We need automated patch and release cycles. So far we've relied on incredibly slow manual processes to accept reports, investigate, verify, patch, and prepare releases. Releasing a fix often takes months. This is way too slow when attackers can just churn out new exploits in hours. We need to iterate on value chain bottlenecks to lower Mean Time To Patch . We should be able to turn around a bug report to a patched pr…

On the other hand, automated fast rollouts leads to a crowdstrike type situation where you brick all the computers of the world immediately. Imo we are going to have to rely on more layers of security. Systems that are designed to be secure even in the presence of individual vulnerabilities. This has already been happening for a while on mobile platforms and game consoles. Even physical hardware designed to keep part…

[deleted]

Re: AI is breaking two vulnerability cultures

#127
post #81

We need automated patch and release cycles. So far we've relied on incredibly slow manual processes to accept reports, investigate, verify, patch, and prepare releases. Releasing a fix often takes months. This is way too slow when attackers can just churn out new exploits in hours. We need to iterate on value chain bottlenecks to lower Mean Time To Patch . We should be able to turn around a bug report to a patched pr…

Sounds like you're expecting the AI-based tools that are finding bugs to also provide fixes. I've been dealing with a bunch of AI-generated (or at least -assisted) vulnerability reports lately. In many cases the reports include proposed patches to fix the issues. It's been..... interesting. In many cases, the analysis provided in the report has been accurate and helpful. In some cases, the proposed patches have also…

Yes, that's why I specified "patched product ready for QA testing". It speeds up the development cycle by making a first pass and ensuring it basically works before passing it to a developer for manual review and a QA tester to ensure the fix doesn't break anything else. Both dev and QA are still in the feedback loop and can make changes until it's ready for release

Re: AI is breaking two vulnerability cultures

#128
post #99

Obviously the solution is for Linux to move to a closed-source development model. Security researchers should report their findings to a committee that includes some big companies (IBM and Oracle seem like trustworthy choices here, but ideally we should find a way to get Microsoft included). Those companies would apply the security patches and distribute binary builds of Linux to their customers. Users fortunate enou…

I like how after so many years, people finally start recognizing that obscurity is a part of security. Not the whole security, obviously, but a part of it.

Just like there's LLM-automated vulnerability fuzzing, there's LLM-automated decompilation. Compilation is no longer a meaningful way to obscure code.

Re: AI is breaking two vulnerability cultures

#129

We need automated patch and release cycles. So far we've relied on incredibly slow manual processes to accept reports, investigate, verify, patch, and prepare releases. Releasing a fix often takes months. This is way too slow when attackers can just churn out new exploits in hours. We need to iterate on value chain bottlenecks to lower Mean Time To Patch . We should be able to turn around a bug report to a patched pr…

On the other hand, automated fast rollouts leads to a crowdstrike type situation where you brick all the computers of the world immediately. Imo we are going to have to rely on more layers of security. Systems that are designed to be secure even in the presence of individual vulnerabilities. This has already been happening for a while on mobile platforms and game consoles. Even physical hardware designed to keep part…

The crowdstrike situation wasn't due to fast rollouts, it was due to a total lack of testing. You can do fast rollouts, with testing, and a mandatory QA signoff. It's called 'continuous delivery' rather than 'continuous deployment'.

I actually don't think more layers of security will fix this. It would be nice if our systems were more secure... but people are, if nothing else, lazy af. Even when adding security isn't a lot of work, people resist it if it "sounds complicated". So I think we're stuck with the status quo. But the big issue now isn't novel bug types, it's the speed in which they're found. Therefore we need to speed up our response.

Re: AI is breaking two vulnerability cultures

#130

Earlier quoted context omitted.

On the other hand, automated fast rollouts leads to a crowdstrike type situation where you brick all the computers of the world immediately. Imo we are going to have to rely on more layers of security. Systems that are designed to be secure even in the presence of individual vulnerabilities. This has already been happening for a while on mobile platforms and game consoles. Even physical hardware designed to keep part…

The crowdstrike situation wasn't due to fast rollouts, it was due to a total lack of testing. You can do fast rollouts, with testing, and a mandatory QA signoff. It's called 'continuous delivery' rather than 'continuous deployment'. I actually don't think more layers of security will fix this. It would be nice if our systems were more secure... but people are, if nothing else, lazy af. Even when adding security isn't…

Lack of care, slip ups, and bugs are basically a constant that will always exist. But we can architect systems which are secure even in the face of bugs. Multiple layers of security mean that even the most critical kernel bug in iOS can never extract your faceid data or encryption key because the hardware physically isn’t capable of it. OSs like Qubes utilise multiple VMs so any kernel bugs have limited reach.

When you look at consoles, they have built software that is resistant to outright glitching the CPU.

Post reply on HN