Related: https://arstechnica.com/gadgets/2025/05/open-source-project-...
XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
51–60 of 128 posts
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#52Earlier quoted context omitted.
J'accuse! You were required to do a paper for BH anyways! :)
Wait a sec, I thought they were optional? > White Paper/Slide Deck/Supporting Materials (optional) > • If you have a completed white paper or draft, slide deck, or other supporting materials, you can optionally provide a link for review by the board. > • Please note: Submission must be self-contained for evaluation, supporting materials are optional. > • PDF or online viewable links are preferred, where no authentica…
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#53Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#54Earlier quoted context omitted.
Whether it is legit-finding is precisely what needs to be checked, but you’re at spot 1061. >130 resolved >303 were classified as Triaged >33 reports marked as new >125 remain pending >208 were marked as duplicates >209 as informative >36 not applicable 20% bind a lot of resources if you have a high input on submissions and the numbers will rise
I think some context I probably don't share with the rest of this thread is that the average quality of a Hacker One submission is incredibly low. Like however bad you think the median bounty submission is, it's worse; think "people threatening to take you to court for not paying them for their report that they can 'XSS' you with the Chrome developer console".
The problem is that the people who know how to use AI properly will slower and more careful in their submissions.
Many others won’t, so we‘ll get lots of noise hiding the real issues. AI makes it easy to produce many bad results in short time.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#55Receiving hundreds of AI generated bug reports would be so demoralizing and probably turn me off from maintaining an open source project forever. I think developers are going to eventually need tools to filter out slop. If you didn’t take the time to write it, why should I take the time to read it?
These aren't like Github Issues reports; they're bug bounty programs, specifically stood up to soak up incoming reports from anonymous strangers looking to make money on their submissions, with the premise being that enough of those reports will drive specific security goals (the scope of each program is, for smart vendors, tailored to engineering goals they have internally) to make it worthwhile.
I think within the next 5 years or so, we are going to see a societal pattern repeating: any program that rewards human ingenuity and input will become industrialized by AI to the point where it becomes a cottage industry of companies flooding every program with 99% AI submissions. What used to be lone wolves or small groups of humans working on bounties will become truckloads of AI generated “stuff” trying to maximize revenue.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#56Earlier quoted context omitted.
That's not their point, I think. They're just saying that those nearly 1060 vulnerabilities are being processed so theirs is being ignored (hence "triage").
If that's all they're saying then there isn't much to do with the sentiment; if you're legit-finding #1061 after legit-findings #1-#1060, that's just life in the NFL. I took instead the meaning that the findings ahead of them were less than legit.
I took instead the opposite - that they were no longer shocked that it was taking so long once they found out why, as they knew who they were and understood.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#57Earlier quoted context omitted.
isnt that called enterprise support / consulting
This is without the enterprise.
but foss is foss, i guess source available doesnt mean we have to read your messages see sqlite (wont even take PR's lol)
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#58Earlier quoted context omitted.
I think some context I probably don't share with the rest of this thread is that the average quality of a Hacker One submission is incredibly low. Like however bad you think the median bounty submission is, it's worse; think "people threatening to take you to court for not paying them for their report that they can 'XSS' you with the Chrome developer console".
We‘ll get this low quality submissions with AI too. The problem is that the people who know how to use AI properly will slower and more careful in their submissions. Many others won’t, so we‘ll get lots of noise hiding the real issues. AI makes it easy to produce many bad results in short time.
Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne
#59First: > To bridge that gap, we started dogfooding XBOW in public and private bug bounty programs hosted on HackerOne. We treated it like any external researcher would: no shortcuts, no internal knowledge—just XBOW, running on its own. Is it dogfooding if you're not doing it to yourself? I'd considerit dogfooding only if they were flooding themselves in AI generated bug reports, not to other people. They're not the o…
Their success rates on HackerOne seem widely varying. 22/24 (Valid / Closed) for Walt Disney 3/43 (Valid / Closed) for AT&T