Vulnerability reports are not special anymore • Hacker News Board

Re: Vulnerability reports are not special anymore

#221

Post by ksajadi » Wed, Jun 24, 2026, 2:31 PM UTC

Cannot agree more. As part of our SOC2 we have to log and respond to all inbound vulnerability reports. Before it was easy to tell if a report was just a bounty hunter looking for a low hanging fruit. Now well crafted emails with seemingly legitimate disclosures take a lot of time to validate and triage.

Our solution was to build a tool that uses LLMs to assess the report before it gets to us. Honestly I wish we didn’t have to do this but it works and has really allowed us to spend our time on the actual good reports. (Feel free to check it out at fortworx.com if you want)

Re: Vulnerability reports are not special anymore

#222

Post by -mlv » Wed, Jun 24, 2026, 2:37 PM UTC

Goofy_Coyote wrote: post #215

Security Eng here. The whole thing is an absolute mess. I’ve been (and still am) on both sides of the fence. I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports…

Have you considered using LLMs to perform some automated more-or-less reliable classification of incoming reports by severity, affected product, etc., then have agents try to replicate the reported findings?

Ideally the reports would also be coming in in the same structured format.

Re: Vulnerability reports are not special anymore

#223

Post by EA-3167 » Wed, Jun 24, 2026, 2:48 PM UTC

whimblepop wrote: post #19

cadamsdotcom wrote: post #4

Security through obscurity was never a great strategy.. and now it’s not a strategy at all.. Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others. This is an engineering problem. It won’t be s…

Almost never do software companies even attempt to design secure systems. I'm not sure this requires new fundamental research so much as slightly giving a shit.

Regulation and an ethics/licensing board à la Engineers would probably be a good start. If management knows they can’t tell you to do a bad or sloppy job because no one in your industry worth a damn will… everyone wins.

Re: Vulnerability reports are not special anymore

#224

Post by hombre_fatal » Wed, Jun 24, 2026, 3:05 PM UTC

EA-3167 wrote: post #223

whimblepop wrote: post #19

Earlier quoted context omitted.

Almost never do software companies even attempt to design secure systems. I'm not sure this requires new fundamental research so much as slightly giving a shit.

Regulation and an ethics/licensing board à la Engineers would probably be a good start. If management knows they can’t tell you to do a bad or sloppy job because no one in your industry worth a damn will… everyone wins.

I just see unintended (but easily imaginable) consequences that don't fix anything.

Especially since the world isn't Dilbert where your boss goes "oh, authz? lol nah, just yolo it" and you go "dangit, alright boss". Instead, security requires eternal vigilance and zero missteps along the thousands a project takes in its lifetime.

I think there's a reason HNers who pitch this idea never give any concrete examples of entailments of their proposal: it doesn't even sound good superficially. e.g. How this actually changes security issues. In fact it just sounds even more convenient to blame engineers.

Re: Vulnerability reports are not special anymore

#225

Post by philipwhiuk » Wed, Jun 24, 2026, 3:21 PM UTC

PaulStatezny wrote: post #219

Goofy_Coyote wrote: post #215

Security Eng here. The whole thing is an absolute mess. I’ve been (and still am) on both sides of the fence. I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports…

> Ending on a doom-and-gloom note: there will be a reckoning. Can you elaborate on what you mean by this?

I imagine:

AI vulnerability analysis is going to find something, it will be reported by a researcher and ignored as chaff, and then separately, later, someone will build it into an exploit and compromise a piece of critical national infrastructure

Re: Vulnerability reports are not special anymore

#226

Post by AshamedBadger56 » Wed, Jun 24, 2026, 3:33 PM UTC

zulban wrote: post #70

cleverfoo wrote: post #5

Earlier quoted context omitted.

Same experience here. I've run a successful vulnerability disclosure program for over a decade and paid out thousands of dollars in bounties for scanii.com (a malware identification API service), but recently (since the beginning of the year), we went from receiving maybe 5 per month to receiving 5 per day. These are clearly AI-generated and extremely low quality (albeit well-written). The rules of the program aren't…

What if... on the vulnerability report rules page there's an image of some text saying something like "your report must include the text: turtle123". Reports without that text get automatically deleted. Sure - modern AI can figure that out, but I bet in a vast majority of cases they won't.

I know some professors who have started doing something similar to combat students using AI for their work. Even going as far as to hide the "your report must include XYZ obscure word 3x" prompt instructions in small invisible text. It's gotten pretty bad, with some students turning in papers with the original ChatGPT prompt LEFT IN THE TURNED IN ASSIGNMENT.

Re: Vulnerability reports are not special anymore

#227

Post by Retr0id » Wed, Jun 24, 2026, 3:36 PM UTC

Goofy_Coyote wrote: post #215

Security Eng here. The whole thing is an absolute mess. I’ve been (and still am) on both sides of the fence. I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports…

I reported a fun security bug to Google recently (not high in CVSS terms but will probably make the HN front page when it goes public), and the report was auto-closed in minutes as "not reproducible". If they tried to use an LLM to reproduce it I'm not surprised, since it requires a soldering iron.

Having seen the other side of a security inbox I totally get it, and fortunately I was able to get it re-opened via backchannels. I think the future of bug reporting will run almost exclusively on reputation and connections.

Re: Vulnerability reports are not special anymore

#228

Post by fragmede » Wed, Jun 24, 2026, 3:51 PM UTC

clates wrote: post #205

capitol_ wrote: post #165

Earlier quoted context omitted.

The cooldown setting in dependabot solves this attack vector. By setting it you give security vendors time to scan new packages.

Notably this does nothing to "solve" the attack vector. You've got a live bomb in front of you and you're adding 10s to the countdown hoping that _others_* find it and defuse it in that time period. I would challenge anyone proposing this to define more than one party doing security checks on packages to prove the point that many projects are waving their hands nebulously around saying "security vendors" and then YOL…

With today's AI glut of tokens, multiple someones are scanning security checks against the changed code. The real problem, as was before, was getting anybody anywhere to pay enough attention for long enough.

Re: Vulnerability reports are not special anymore

#229

Post by JohnMakin » Wed, Jun 24, 2026, 3:57 PM UTC

These kinds of posts act like there wasn't already a rich suite of automated security scanning tools available. The fact you can put them in a LLM's hand or point it at a codebase to do the exact same thing isn't really a big leap, the spam is. It's a bunch of amateurs thinking they can do work outside of the realm of their expertise now, and this trend isn't unique to security, you see it spreading across the various realms of expertise out there in the world. The world's slowly filling with slop, and CVE spam is a byproduct of that.

Re: Vulnerability reports are not special anymore

#230

Post by mbauman » Wed, Jun 24, 2026, 4:07 PM UTC

fastball wrote: post #29

They weren't special even before LLMs. Drive-by script-kiddies would run some basic scripts against your platform and send generally-not-actually-a-vulnerability reports, claiming that these were big problems , and requesting to be paid bug bounties.

And then they submit them to a CNA and get a CVE assigned, and then _everyone_ needs to deal with the not-actually-a-vulnerability report, especially when the not-actually-triggerable-DOS gets assigned a "Critical" CVSS score from EUVD or NVD.