Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are retiring our bug bounty program

turso.tech

Post reply on HN
Print view View original

171–180 of 303 posts

Re: We are retiring our bug bounty program

#171

Post by skydhash »

CharlesW wrote: post #78
limaoscarjuliet wrote: post #27

Earlier quoted context omitted.

"[...] bottleneck isn't in writing the code. It is in reading and understanding the code". 100% agreed! Furthermore, the more code is generated by AI, the fewer people will actually understand it!

Generally, software engineers already have little to no understanding of the code that's actually being executed. We're so used to high- and higher-level abstractions like C, Go, Python, and JavaScript that we forget that we're already working with mostly-deterministic symbolism in a process that more closely resembles invoking magic spells than writing machine code. One more level of abstraction is not the end of so…

The thing is that C is formal by itself. Opcodes, Assembly, C, Python, Common Lisp,… are all equivalent to each other, meanings there’s no statement and no algorithm that you can’t map to each other. That’s what it means to be Turing Complete.

The main issue is that not everyone cares about the semantic of what they’re writing. You don’t need to know assembly to talk about C’s semantic or know C to talk about Python semantics. It does not require going up and down some abstraction tower.

Re: We are retiring our bug bounty program

#172

Post by overgard »

jquery wrote: post #19
ToucanLoucan wrote: post #7

Oh look it's more of exactly what AI skeptics said would happen: low effort bullshit generated at scale making life hell for people actually trying to make things. That's wild. Edit: it is genuinely wild, I don't know of another product category that selects so perfectly for the WORST type of person to be it's enthusiast. Just every single person I see hyped about AI is fucking insufferable on at least one and usuall…

I think people would be more interested in listening to "AI skeptics" if they offered realistic solutions to the problems they predict. Pandora's box has been opened, let's deal with the consequences now instead of trying to shut the box which cannot be shut.

"We created a massive problem but it's up to you, the people that never wanted this, to clean up our mess."

Re: We are retiring our bug bounty program

#173

Post by thfuran »

hunterpayne wrote: post #163
CharlesW wrote: post #78

Earlier quoted context omitted.

Generally, software engineers already have little to no understanding of the code that's actually being executed. We're so used to high- and higher-level abstractions like C, Go, Python, and JavaScript that we forget that we're already working with mostly-deterministic symbolism in a process that more closely resembles invoking magic spells than writing machine code. One more level of abstraction is not the end of so…

That's a you problem. If you feel this way, its the universe saying that you aren't very good at writing software. Good engineers don't have this problem. PS We have way too many levels of abstraction now, that doesn't mean the right answer is to add another. Even worse unlike the others, LLMs aren't deterministic.

[dead]

Re: We are retiring our bug bounty program

#174

Post by skydhash »

ijk wrote: post #107
dogleash wrote: post #97

Earlier quoted context omitted.

To the extent that's true, it's already a problem plaguing the profession. I wouldn't advocate for using different tools, but everyone should be able to reason about the machine instructions underlying their code. Both in the immediate sense of the assembly a simple function turns into, and the tricks language runtimes use to enable their neat features. The attitude that things are magic is poison. There is a differe…

I agree in principle, but every time I run a debugger on modern C++ it makes it clear that, rather than being a simple and cutesy transformation, "compiler optimization" is actually black magic.

It’s not. You “only” have to learn about computer architecture and computation theory. Meaning a lot of maths /s

Re: We are retiring our bug bounty program

#175

Post by [deleted] »

CamperBob2 wrote: post #129
joe_mamba wrote: post #90

Earlier quoted context omitted.

But it really doesn't have to be like this. For their bug bounty program, the company can just charge 5-10$ per submission to guarantee everything you send gets thoroughly reviewed by a human, and so it completely eliminates bot slop DDoS submissions overnight. If your bug and PR was actually good, then you get 10 + 1000$ back, and if it wasn't good, then you need to do better due diligence next time, and the skilled…

I said it before and I'll say it again, for opportunities open to the entire world on the internet, adding monetary friction is the only way to filter out serious people from bad actors doing spray-and-pray hoping they make some money or get that job through weaponizing AI bots and sucking all the air in the room. So many problems can be solved that way, including customer support. Instead of having to post a sob sto…

[deleted]

Re: We are retiring our bug bounty program

#177

Post by miyoji »

ToucanLoucan wrote: post #148
miyoji wrote: post #75

Earlier quoted context omitted.

This response is incredibly annoying and insufferable. It's only "impossible" at this point because people continually ignored skeptics and anyone warning about exactly these outcomes. Now that doom is here, it's too late to do anything about it. Just accept the doom!

My only objection is calling it doom. It isn't and calling it that gives this stupid shit used by low-effort people far, far too much credit. It's just slop. But it does suck, you know? Part of what makes OSS so great is that anyone could contribute. If someone uses a thing, and finds something broken or a way to make it better, they could do that and then push it back up to the project and ideally have it merged so…

I mean, I was being a little hyperbolic and spitting back against people who claim that any criticism of AI is "doomerism", but I edited out the part of my comment that made that clear before posting. My bad.

Although I do want to push back mildly. I think this situation is a bit worse than just "it sucks", and if you extrapolate out to a world where every institution that's like open source gets polluted by the same fundamental dynamics, it's not quite doom, but it's quite a bit worse than "it does suck".

Re: We are retiring our bug bounty program

#178

Post by 48terry »

caymanjim wrote: post #98
pscanf wrote: post #59

We sorely need a way to reliably detect AI slop, but unfortunately it doesn't seem possible and it's just getting harder and harder. Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly…

There's no reason to care that a human spent time on it. Humans are bad at writing code. Garbage PRs and slop have been a problem in open source and bug bounty programs since long before AI came on the scene. We need better AI so that there's no need to solicit external bug fixes, and better AI so other contributions can be evaluated for usefulness and quality. What do you care if a human ever looked at it at all? It…

If AI is already mass-producing garbage PRs and other unreliable crap, what makes AI (established as producing unreliable crap) the solution for review? What makes the reviewing AI not produce unreliable crap with regards to the review?

A magical, hypothetical AI that always gets it right and will make all these problems go away is neither a solution nor a plan. It's wishful thinking.

Re: We are retiring our bug bounty program

#179

Post by intrasight »

KronisLV wrote: post #74
user_7832 wrote: post #37

Earlier quoted context omitted.

Honestly I think this is a great idea. My only suggestion is instead of being very nominal, it should be "reasonable" (so $10 and not $1). It's even possible to directly link this to maintainers/employees - if you can review 10 such AI/real things per hour (likely more if it's AI slop that's easy to detect), you're generating another revenue stream. Now, I have no idea if these guys are based in SF Bay or a 3rd world…

Sounds like a startup idea to me! Admittedly, the friction and the fact that you have to pay would prevent a lot of legitimate people from participation which sucks. AI is really throwing a wrench in the economics of software development, isn’t it?

$10 or even $100 should create no such friction on a $1000 bounty.

Re: We are retiring our bug bounty program

#180

Post by Dove »

Let us imagine a Bug Bounty Bouncer Service.

The project does not accept bug bounty submissions without BBBS attestation. To get it, you must first submit your report to the BBBS for review.

Now, if this is your first submission (you are unknown to the BBBS), you must submit $50 to the BBBS along with the bug report, to pay a human to spend an hour looking at your work to verify it is written in good faith. This is not a review of whether the bug is real or valuable, just a readover to verify the report is coherent and plausible. If you have done this before, you can get a free attestation based on being a member in good standing, but submitting slop (per the judgement of the BBBS reviewer or the project receiving the report) is an account ban.

The BBBS couldn't steal your work and submit it themselves if they gave you some sort of signed hash as a receipt, which as a side effect would also be a deterrant against bounty programs stealing your work.

Submissions would only be expensive per submission for an anonymous user, enabling the low friction high trust communication under which collaboration works best when reputation has been established.

The BBBS itself won't be overrun by slop since the price of establishing an account far exceeds what a bot might expect to make with a single malicious submission. Nor can legitimate established accounts be sold since the cost of creating them exceeds the value to be expected from abusing them. Moreover, the cost to establish a reputation as a bug bounty hunter is small in dollars compared to the cost in time and expertise that a legitimate hunter would be expected to expend in the course of their work.

The vast majority of slop would go away as the cost of a first submission is much too high. The cost to the project is close to nothing - integrating with the BBBS attestation API. The cost to a legitimate bug bounty hunter is low - some human review while establishing a reputation, which could even be made useful if it came in the form of feedback. All review is paid for by the submitter, so no one is trying to counter infinite slop with volunteer hours.

Moreover, the BBBS can serve as a mediator of trust, not only against AI, but as a place to receive reputational merit for high value work and trustworthy bug bounty programs.

I realize I am describing a lightweight guild, which is subject to well known political failure modes (the most significant of which is exploiting newcomers), but the concept has the advantage that guilds have functioned as successful slop gatekeepers in society for a very long time and a lot is known about how to make them work.

Post reply on HN