Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulnerability reports are not special anymore

words.filippo.io

Post reply on HN
Print view View original

151–160 of 249 posts

Re: Vulnerability reports are not special anymore

#151

Post by pixlmint »

socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

> Once most of the bugs are fixed

my brother in christ, I hope you're actually trying to be funny here.

Re: Vulnerability reports are not special anymore

#152

Post by sass_muffin »

StrauXX wrote: post #148
jamesfinlayson wrote: post #62

Earlier quoted context omitted.

Yep, I've updated dependencies with an RCE that can't be exploited in my codebase just to keep my security team happy. Not worth the multiple arguments about it not actually being an issue.

You can never guarantee that the codepath of a dependency that is vulnerable can not be reached or used as a gadget in an exploit chain. Patching dependencies, even when no direct vulnerability arises is an essential part of defense in depth and sevurity hygene.

You can also never guarantee the patched software doesn't include a worse vulnerability, I would submit that patching software without proper time to validate changes is also a security issue.

If you aren't careful, that is how you get this security theater.

Re: Vulnerability reports are not special anymore

#153

Post by KptMarchewa »

froddd wrote: post #104
Gigachad wrote: post #90

Earlier quoted context omitted.

It's easier to keep stuff up to date these days. If you have a project with typescript, unit tests, and end to end tests like cypress you can just have dependabot create the PRs to update packages. If everything passes you just have to hit the merge button. Just updating everything is probably easier than assessing if it's possible to trigger an exploit with the way you use the package.

This is exactly how developers of malware want you to behave. Update without really thinking about it. I do wonder how long it will take before an attack is developed by submitting a semi-genuine vulnerability, shortly followed by a ‘fix’ including malicious code.

Dependency cooldowns fix most of those problems.

Re: Vulnerability reports are not special anymore

#154

Post by himata4113 »

socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

This is where you're wrong, I ran an experiment and told it to find bugs in a ~200 LoC project. The models are tuned in a way to where they're expected to generate issue reports so a codebase that had zero bugs, zero vulnerabilities and zero changes needed it found 3 low severity issues (cosmetic) 1 medium severity issue and 1 critical severity issue. The critical severity issue was accepting unvalidated user input, for... an echo command.

Re: Vulnerability reports are not special anymore

#157

Post by wobbat »

zadg wrote: post #156

I think the main impact of this is that a successful career in vulnerability research is going to require a high level of proficiency in exploit development, as that's where the demonstrable real-world impact lies.

Honestly, even that is something LLMs are becoming scarily good at. Not all of it, but they have surprised me, including in terms of exploit development, more than I am comfortable with.

Re: Vulnerability reports are not special anymore

#159

Post by muldvarp »

zadg wrote: post #156

I think the main impact of this is that a successful career in vulnerability research is going to require a high level of proficiency in exploit development, as that's where the demonstrable real-world impact lies.

Tech careers no longer exist. Tech jobs will still exist for a few years, but careers they will no longer be.

Re: Vulnerability reports are not special anymore

#160

Post by cadamsdotcom »

jstimpfle wrote: post #147
cadamsdotcom wrote: post #4

Security through obscurity was never a great strategy.. and now it’s not a strategy at all.. Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others. This is an engineering problem. It won’t be s…

It might just mean the opposite. Unergonomic and/or slow memory safe languages might not be needed anymore at some point, because LLM can check for reasonable programming patterns and can do some amount of vulnerability checking upfront. But seriously the first point -- as long as you keep to a known set of reasonable patterns (much larger set than the sets accepted by restrictive and cumbersome type systems), memory…

> can reliably find some

Some.

In software & in security, 99% is a failing grade.

So is 99.99%, so is 99.999%, and any other amount less than 100%. It’s not enough to point 5 LLMs at it and it’s not enough to point 500 LLMs at it.

The field needs to seek deterministic & comprehensive solutions to whole problem classes.

Post reply on HN