Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

xbow.com

Post reply on HN
Print view View original

121–128 of 128 posts

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#121

Post by monster_truck »

ackbar03 wrote: post #108
monster_truck wrote: post #74

Earlier quoted context omitted.

Walt Disney's program covers substantially more surface area, there's 6? publicly traded companies listed there. In addition to covering far fewer domains & apps, AT&T's conditions and exclusions disqualify a lot more. The market for bounties is a circus, breadcrumbs for free work from people trying to 'make it'. It can safely be analogized to the classic trope of those wanting to work in games getting paid fractiona…

> The market for bounties is a circus, breadcrumbs for free work from people trying to 'make it'. > The number of CVSS vulns with a score above 8 that have floated across the front page of HN in the past year without anyone getting paid tells you that much. You make it sound like there's a ton of people going around who can just dig up CVSS vulns above 8 and is making me all confused. Is that really happening? I have…

The weighted average is 7.6. Finding them doesn't necessarily take much effort if you know what to look for.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#122

Post by lallysingh »

skeptrune wrote: post #91

I'm confused on whether or not this actually outperformed humans. The more interesting statistic would be how much money it made versus the average hacker one top ranked contributor.

I think a better one would be the average compute cost for xbow per exploit found, if you're interested in the shift in security economics this represents.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#123

Post by keisborg »

8200_unit wrote: post #120
keisborg wrote: post #115

«XBOW submitted nearly 1,060 vulnerabilities. All findings were fully automated, though our security team reviewed them pre-submission to comply with HackerOne’s policy on automated tools» That seems a bit unethical. I’ve thought companies specifically deny usage of automated tools. A bit too late ey…?

They acknowledge that in the article and all submissions are human reviewed before they are submitted.

The policies states it’s not allowed to use automated tools, not to submit report using automated tools alone. Human review does not really change that.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#124

Post by tptacek »

billy99k wrote: post #114
t0mas88 wrote: post #72

Earlier quoted context omitted.

Might be fixable by adding a $ 100 submission fee that is returned when you're proving working exploit code. Would make the Curl team a lot of money.

I've been on Hackerone for almost 8 years and I think the problem with this is that too many companies won't pay for legitimate bugs, even when you have a working exploit. I had one critical bug take 3 years to get a pay out. I had a full walkthrough with videos and report. The company kept stalling and at one point told me that because they completely had the app remade, they weren't going to pay me anything. Hacker…

I do think HackerOne is problematic, in that it pushes companies that don't really understand bug bounties to stand up bounty programs without a clear reason. If you're doing a serious bounty, your incentive is to pay out. But a lot of companies do these bounties because they just think they're supposed to.

Most companies should not do bug bounties.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#125

Post by beng-nl »

err4nt wrote: post #14
jgalt212 wrote: post #9

Earlier quoted context omitted.

One would think if AI can generate the slop it could also triage the slop.

How does it know the difference?

This might not always work, but whenever possible, a working exploit could be demanded, working in a form that can be automatically verified to work.

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#126

Post by slt2021 »

keisborg wrote: post #123
8200_unit wrote: post #120

Earlier quoted context omitted.

They acknowledge that in the article and all submissions are human reviewed before they are submitted.

The policies states it’s not allowed to use automated tools, not to submit report using automated tools alone. Human review does not really change that.

if a human reviewer can repro the bug, there is no difference between automated or human found bug.

bug works and is repro - as a software owner, do you care if human or ai found it?

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#127

Post by keisborg »

slt2021 wrote: post #126
keisborg wrote: post #123

Earlier quoted context omitted.

The policies states it’s not allowed to use automated tools, not to submit report using automated tools alone. Human review does not really change that.

if a human reviewer can repro the bug, there is no difference between automated or human found bug. bug works and is repro - as a software owner, do you care if human or ai found it?

I cannot answer for all the program owners, but I imagine that there are other concerns than reproducibility

Re: XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

#128

Post by vmayoral »

vasco wrote: post #97
vmayoral wrote: post #95

It’s humans who: - Design the system and prompts - Build and integrate the attack tools - Guide the decision logic and analysis This isn’t just semantics — overstating AI capabilities can confuse the public and mislead buyers, especially in high-stakes security contexts. I say this as someone actively working in this space. I participated in the development of PentestGPT, which helped kickstart this wave of research…

> It's humans Who would it be, gremlins? Those humans weren't at the top of the leaderboard before they had the AI, so clearly it helps.

Actually, those humans (XBOW's) were already top rankers. Just look it up.

What's being critized here is the hype, which can be misleading and confusing. On this topic, wrote a small essay: “Cybersecurity AI: The Dangerous Gap Between Automation and Autonomy,” to sort fact from fiction -> https://shorturl.at/1ytz7

Post reply on HN