Live data from Hacker News

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

xbow.com

51–60 of 128 posts

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#51
post #3

Related: https://arstechnica.com/gadgets/2025/05/open-source-project-...

The main difference is that all of the vulnerabilities reported here are real, many quite critical (XXE, RCE, SQLi, etc.). To be fair there were definitely a lot of XSS, but the main reason for that is that it's a really common vulnerability.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#52
post #48
post #47

Earlier quoted context omitted.

J'accuse! You were required to do a paper for BH anyways! :)

Wait a sec, I thought they were optional? > White Paper/Slide Deck/Supporting Materials (optional) > • If you have a completed white paper or draft, slide deck, or other supporting materials, you can optionally provide a link for review by the board. > • Please note: Submission must be self-contained for evaluation, supporting materials are optional. > • PDF or online viewable links are preferred, where no authentica…

I think you're fine, most people don't take the paper bit seriously. It's not due until the end of July regardless (you don't need a paper to submit for the CFP).

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#53
post #13

Earlier quoted context omitted.

Eventually projects who can afford the smugness are going to charge people to be able to talk to open source developers.

isnt that called enterprise support / consulting

This is without the enterprise.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#54
post #41
post #39

Earlier quoted context omitted.

Whether it is legit-finding is precisely what needs to be checked, but you’re at spot 1061. >130 resolved >303 were classified as Triaged >33 reports marked as new >125 remain pending >208 were marked as duplicates >209 as informative >36 not applicable 20% bind a lot of resources if you have a high input on submissions and the numbers will rise

I think some context I probably don't share with the rest of this thread is that the average quality of a Hacker One submission is incredibly low. Like however bad you think the median bounty submission is, it's worse; think "people threatening to take you to court for not paying them for their report that they can 'XSS' you with the Chrome developer console".

We‘ll get this low quality submissions with AI too.

The problem is that the people who know how to use AI properly will slower and more careful in their submissions.

Many others won’t, so we‘ll get lots of noise hiding the real issues. AI makes it easy to produce many bad results in short time.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#55
post #44

Receiving hundreds of AI generated bug reports would be so demoralizing and probably turn me off from maintaining an open source project forever. I think developers are going to eventually need tools to filter out slop. If you didn’t take the time to write it, why should I take the time to read it?

These aren't like Github Issues reports; they're bug bounty programs, specifically stood up to soak up incoming reports from anonymous strangers looking to make money on their submissions, with the premise being that enough of those reports will drive specific security goals (the scope of each program is, for smart vendors, tailored to engineering goals they have internally) to make it worthwhile.

Got it! The financial incentive will probably turn out to be a double edged sword. Maybe in the pre-AI age, it’s By Design to drive those goals, but I bet the ability to automate submissions will inevitably alter the rules of these programs.

I think within the next 5 years or so, we are going to see a societal pattern repeating: any program that rewards human ingenuity and input will become industrialized by AI to the point where it becomes a cottage industry of companies flooding every program with 99% AI submissions. What used to be lone wolves or small groups of humans working on bounties will become truckloads of AI generated “stuff” trying to maximize revenue.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#56
post #27

Earlier quoted context omitted.

That's not their point, I think. They're just saying that those nearly 1060 vulnerabilities are being processed so theirs is being ignored (hence "triage").

If that's all they're saying then there isn't much to do with the sentiment; if you're legit-finding #1061 after legit-findings #1-#1060, that's just life in the NFL. I took instead the meaning that the findings ahead of them were less than legit.

> I took instead the meaning that the findings ahead of them were less than legit.

I took instead the opposite - that they were no longer shocked that it was taking so long once they found out why, as they knew who they were and understood.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#57
post #13

Earlier quoted context omitted.

isnt that called enterprise support / consulting

This is without the enterprise.

gotchu, maybe i could see github donations enabling issue creation or wahtever in the future idk

but foss is foss, i guess source available doesnt mean we have to read your messages see sqlite (wont even take PR's lol)

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#58
post #54
post #41

Earlier quoted context omitted.

I think some context I probably don't share with the rest of this thread is that the average quality of a Hacker One submission is incredibly low. Like however bad you think the median bounty submission is, it's worse; think "people threatening to take you to court for not paying them for their report that they can 'XSS' you with the Chrome developer console".

We‘ll get this low quality submissions with AI too. The problem is that the people who know how to use AI properly will slower and more careful in their submissions. Many others won’t, so we‘ll get lots of noise hiding the real issues. AI makes it easy to produce many bad results in short time.

Everyone already agrees with that; the interesting argument here is that it also makes it easy to produce many good results in short time.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#59

First: > To bridge that gap, we started dogfooding XBOW in public and private bug bounty programs hosted on HackerOne. We treated it like any external researcher would: no shortcuts, no internal knowledge—just XBOW, running on its own. Is it dogfooding if you're not doing it to yourself? I'd considerit dogfooding only if they were flooding themselves in AI generated bug reports, not to other people. They're not the o…

Their success rates on HackerOne seem widely varying. 22/24 (Valid / Closed) for Walt Disney 3/43 (Valid / Closed) for AT&T

Walt Disney doesn't pay bug bounties. AT&T's bounties go up to $5k, which is decent but still not much. It's possible that the market for bugs is efficient.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#60
I'm generally pretty bearish on AI security research, and think most people don't know anything about what they're talking about, but XBOW is frankly one of the few legitimately interesting and competent companies in the space, and their writeups and reports have good and well thought out results. Congrats!
Post reply on HN