Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are retiring our bug bounty program

turso.tech

Post reply on HN
Print view View original

81–90 of 303 posts

Re: We are retiring our bug bounty program

#81

Post by jcgrillo »

dale_glass wrote: post #63
jcgrillo wrote: post #55

Earlier quoted context omitted.

> forget about the shutting it down and think of something actually realistic. Why is it not realistic? Small teams do excellent work. Keep your team small and trusted. Only accept contributions from your team, and people outside your team who are personally vouched for by someone on your team. It's like climbing mountains or sailing or any other type of inherently risky activity--you don't go out with people you don…

That's not shutting anything down, that's just being selective with what you accept, and everyone did that already to some extent. Even pre-AI it was obvious that contributions have to be vetted for a bunch of reasons.

Right, so the Github "open contributions" model where anyone can open an issue or a PR or otherwise waste a maintainer's time is broken. Fundamentally insecure under this type of attack. Now that the exploit is being used widely, and costing us immensely, we need to put a lid on it. If the only way to guarantee an AI bot (or its meatspace sock puppet) doesn't waste your time is to move to a "look but don't touch" model, then that's what we need to do. I think this would be a reasonable default:

Public repos are read only except for contributors who have been given specific permission, and those permissions are granular e.g. in order of increasing damage potential:

- comment on issue

- create issue

- comment on PR

- create PR

- run CI against PR

- etc.

In other words, shut it down.

Re: We are retiring our bug bounty program

#82

Post by mackeye »

curtisblaine wrote: post #62

Bots are using real tokens for this. So, ultimate honeypot idea: post heavily commented skeleton code in a github repo, promise a generous money reward for closing issues and never pay anyone. See the bots swarm and burn their tokens to write code for you.

:D https://github.com/UnsafeLabs/Bounty-Hunters

Re: We are retiring our bug bounty program

#83

Post by bee_rider »

MostlyStable wrote: post #16

Closing the program is totally reasonable. However, there is another option: Make submitters pay a nominal fee that is returned in the case that a real bug is found.

It sounds like the bug bounty requires the user to extend the simulator, to cover the type of bug they found. Maybe the they could require a full run of the simulator test suite before submission? This serves as a nice check (that they didn’t break the simulator), and maybe it could also produce some proof-of-work artifact as a side-effect… (is this possible? I don’t know security).

Re: We are retiring our bug bounty program

#84

Post by pdimitar »

satvikpendem wrote: post #73
sparklingmango wrote: post #51

Earlier quoted context omitted.

Power dynamics. Usually the person making the giant PRs is the one with all the sway. An earlier-career engineer is unlikely to push back against that level of influence.

It can be a company wide policy rather than trying to target a single individual even if the outcome is that they are targeted. This is something that should be addressed to them through a manager etc or if not, it's time to leave while they ruin the product over time.

This "you should leave" thing is a very boring and tired take and it should be said regularly that almost no engineer can afford it nowadays.

Beautiful theory, but only that.

Re: We are retiring our bug bounty program

#85

Post by duskdozer »

jcgrillo wrote: post #81
dale_glass wrote: post #63

Earlier quoted context omitted.

That's not shutting anything down, that's just being selective with what you accept, and everyone did that already to some extent. Even pre-AI it was obvious that contributions have to be vetted for a bunch of reasons.

Right, so the Github "open contributions" model where anyone can open an issue or a PR or otherwise waste a maintainer's time is broken. Fundamentally insecure under this type of attack. Now that the exploit is being used widely, and costing us immensely, we need to put a lid on it. If the only way to guarantee an AI bot (or its meatspace sock puppet) doesn't waste your time is to move to a "look but don't touch" mod…

I think I saw this on here yesterday: https://github.com/mitchellh/vouch

Not great for privacy or ad-hoc contributions, but I don't see a way out of the muck without some kind of trust net.

Re: We are retiring our bug bounty program

#86

Post by wg0 »

chapinb wrote: post #17
wg0 wrote: post #8

Which goes on to prove that bottleneck isn't in writing the code. It is in reading and understanding the code. We all had that one "productive" engineer in our teams who would write huge PRs that would have large swaths of refactoring whether warranted or not and that was way before anyone even could imagine in their wildest dreams that neural networks could generate that huge amounts of code. The net effect of such…

Sounds a like a tactical tornado, made me think of this paragraph: “Almost every software development organization has at least one developer who takes tactical programming to the extreme: a tactical tornado. The tactical tornado is a prolific programmer who pumps out code far faster than others but works in a totally tactical fashion. When it comes to implementing a quick feature, nobody gets it done faster than the…

This is profound and beautiful description. Thank you for sharing. Totally can relate to that. Been there, seen that.

Re: We are retiring our bug bounty program

#87

Post by frakt0x90 »

singpolyma3 wrote: post #66

It's a bit odd that this comes today after so many other projects reverse this finding.

AI can find useful exploits but the highly publicized ones are among a sea of false positives and the successes I've read were found by people who were already experts. I can 100% see a public bug bounty program being inundated with garbage even if there are diamonds in the rough.

Re: We are retiring our bug bounty program

#88

Post by malfist »

KolmogorovComp wrote: post #71
MostlyStable wrote: post #16

Closing the program is totally reasonable. However, there is another option: Make submitters pay a nominal fee that is returned in the case that a real bug is found.

Unfortunately this isn't all black-and-white. There are some bug bounty where the company is very eager not to pay any bounty, aggressively marking vulnerabilities as out-of-scope or working-as-intended. In those case you already lose time, but in the future you would also lose money. Unfortunately you don't know how a company will react before submitting, especially if it's a small one.

It already doesn't stand on face value. These people are spending money to open PRs via their token costs

Re: We are retiring our bug bounty program

#89

Post by wg0 »

jt2190 wrote: post #40
Neywiny wrote: post #20

Earlier quoted context omitted.

I was (almost) just that guy for one PR. Removed something like 20% or more of the codebase by leveraging the libraries and external tools we already had in use better, but it meant almost every single thing we were doing had to use the library function instead of the one we wrote. But assuming you have good regression tests and linters, so you know the code works and it's not terrible, the review should be more abou…

You’re not an example of what we’re taking about here. Congratulations! A better example would be if you’d changed the behavior of the library as you did this work, and the library changes introduced hard-to-detect bugs across the application.

Yes exactly. the GP isn't what we are talking about it and huge PR isn't what we are talking about either.

PR can be huge that's OK. For example, codebases that moved from Python 2 to Python 3 would have had huge PRs but the cognitive load was well understood.

Re: We are retiring our bug bounty program

#90

Post by joe_mamba »

abirch wrote: post #25
chapinb wrote: post #17

Earlier quoted context omitted.

Sounds a like a tactical tornado, made me think of this paragraph: “Almost every software development organization has at least one developer who takes tactical programming to the extreme: a tactical tornado. The tactical tornado is a prolific programmer who pumps out code far faster than others but works in a totally tactical fashion. When it comes to implementing a quick feature, nobody gets it done faster than the…

AI can be the ultimate tactical tornado.

But it really doesn't have to be like this.

For their bug bounty program, the company can just charge 5-10$ per submission to guarantee everything you send gets thoroughly reviewed by a human, and so it completely eliminates bot slop DDoS submissions overnight. If your bug and PR was actually good, then you get 10 + 1000$ back, and if it wasn't good, then you need to do better due diligence next time, and the skilled human feedback you received on why it wasn't good, was a valuable lesson for your engineering career, and it only cost you the price of a Starbucks latte, and it also cut out all the scammers polluting the system. This way everyone wins.

I said it before and I'll say it again, for opportunities open to the entire world on the internet, adding monetary friction is THE ONLY (anonymous) WAY to filter out serious people from bad actors doing spray-and-pray hoping they'll make some money, or get that job, by weaponizing AI bots. You can't rely on honor systems and a high trust society on the anonymous open internet, you need to financially gatekeep to save yourself and your sanity, and make sure the honest serious people you want to engage with don't end up drowning in the noise of the scammers and unscrupulous opportunists.

But we can't shut ourselves down just because we refuse to apply solutions to AI slop DDoS.

Post reply on HN