All the fun vanishes.
Good. It was in the way.
XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
21–30 of 128 posts
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#22Earlier quoted context omitted.
One would think if AI can generate the slop it could also triage the slop.
How does it know the difference?
I'm still on the AI-skeptic side of the spectrum (though shifting more towards "it has some useful applications"), but, I think the easy answer is - if different models/prompts are used in generation than in quality-/correctness-checking.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#23> XBOW submitted nearly 1,060 vulnerabilities. Yikes, explains why my manually submitted single vulnerability is taking weeks to triage.
The XBOW people are not randos.
That's not their point, I think. They're just saying that those nearly 1060 vulnerabilities are being processed so theirs is being ignored (hence "triage").
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#24First:
> To bridge that gap, we started dogfooding XBOW in public and private bug bounty programs hosted on HackerOne. We treated it like any external researcher would: no shortcuts, no internal knowledge—just XBOW, running on its own.
Is it dogfooding if you're not doing it to yourself? I'd considerit dogfooding only if they were flooding themselves in AI generated bug reports, not to other people. They're not the ones reviewing them.
Also, honest question: what does "best" means here? The one that has sent the most reports?
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#25Receiving hundreds of AI generated bug reports would be so demoralizing and probably turn me off from maintaining an open source project forever. I think developers are going to eventually need tools to filter out slop. If you didn’t take the time to write it, why should I take the time to read it?
Open source maintainers have been complaining about this for a while. https://sethmlarson.dev/slop-security-reports. I'm assuming the proliferation of AI will have some significant changes on/already has had for open source projects.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#26"XBOW is an enterprise solution. If your company would like a demo, email us at info@xbow.com."
Like any "AI" article, this is an ad.
If you are willing to tolerate a high false positive rate, you can as well use Rational Purify or various analyzers.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#27Earlier quoted context omitted.
The XBOW people are not randos.
That's not their point, I think. They're just saying that those nearly 1060 vulnerabilities are being processed so theirs is being ignored (hence "triage").
If that's all they're saying then there isn't much to do with the sentiment; if you're legit-finding #1061 after legit-findings #1-#1060, that's just life in the NFL. I took instead the meaning that the findings ahead of them were less than legit.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#28Earlier quoted context omitted.
HackerOne was already useless years before LLMs. Vulnerability scanning was already automated. When we put our product on there, roughly 2019, the enterprising hackers ran their scanners, submitted everything they found as the highest possible severity to attempt to maximize their payout, and moved on. We wasted time triaging all the stuff they submitted that was nonsense, got nothing valuable out of the engagement,…
Moreover, I don't think XBOW is likely generating the kind of slop beg bounty people generate. There's some serious work behind this.
Still they're sending hundreds of reports that are being refused because they are not following the rules of the bounties. So they better work on that.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#29Earlier quoted context omitted.
HackerOne was already useless years before LLMs. Vulnerability scanning was already automated. When we put our product on there, roughly 2019, the enterprising hackers ran their scanners, submitted everything they found as the highest possible severity to attempt to maximize their payout, and moved on. We wasted time triaging all the stuff they submitted that was nonsense, got nothing valuable out of the engagement,…
Moreover, I don't think XBOW is likely generating the kind of slop beg bounty people generate. There's some serious work behind this.
Do you have sources for if we want to learn more?