Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT‑5.5 Bio Bug Bounty

openai.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: GPT‑5.5 Bio Bug Bounty

#81

Post by IAmGraydon »

chromacity wrote: post #71
altcognito wrote: post #29

Billions upon billions going to these companies. 25k reward from a selected group of people if you help us determine whether or not someone can use our tool to generate weapons of mass destruction.

Because it can't and it's a publicity stunt. It achieves three goals: 1) Underscores to the general public that the models are amazingly powerful and if you're not using them, your competitors will out-innovate you, 2) Sends the message to regulators that they don't need to do anything because the companies are diligent to prevent harm, 3) Sends the message to regulators that they sure should be regulating "open-sour…

I’m glad people are starting to recognize this, but when will the general public? Never?

Re: GPT‑5.5 Bio Bug Bounty

#82

Post by stratos123 »

stonogo wrote: post #43
dist-epoch wrote: post #30

Earlier quoted context omitted.

This model is much more powerful than gpt-oss-20b, notice how the contest was not even for the 120b model. Also, bio was not a subject.

The model is more powerful, so the bounty is 1/20th the size? More risk, less reward? "Biorisk" seems to be a concept not only invented by OpenAI but exclusively taken seriously by them. I wonder if this program is less about finding actual risks than it is hopefully casting a wide net for someone to help them prove their model is relevant in this space.

> "Biorisk" seems to be a concept not only invented by OpenAI but exclusively taken seriously by them.

This is false. Antropic just bundles it into CBRN. As for inventing it, the idea of AI-created bioweapons as a concrete risk far predates OpenAI as a company.

Re: GPT‑5.5 Bio Bug Bounty

#83

Post by dist-epoch »

hyperpape wrote: post #54
dist-epoch wrote: post #31

Earlier quoted context omitted.

Because the questions themselves are dangerous. Probably along the lines of "how would you create a small biolab for virus research in a kitchen with $20k?" or "how do I take the DNA sequence from https://www.ncbi.nlm.nih.gov/nuccore/NC_001611.1 and assemble it?"

Which is difficult, because the fact that you can come up with your example questions tells us they're probably not very dangerous. Plenty of ink has been spilled about how LLMs could help people create bioweapons. The basic idea "you could do dangerous things with an LLM" is already pop culture, and you're not doing anything dangerous by giving easy example questions. A dangerous question would have to be along the…

> because the fact that you can come up with your example questions tells us they're probably not very dangerous

maybe I know more about this field that you think

there are biologists on video saying that present day models have expert level wet-lab knowledge and can guide a novice through whole procedures

models also were able to tweak DNA sequences to make them bypass DNA-printing companies filters

> they don't want bad publicity when someone not under NDA jailbreaks their model and answers their question

just like people now pay $500k for Chrome vulnerabilities, soon people will pay similar amounts to jailbrake models to do bad things

Re: GPT‑5.5 Bio Bug Bounty

#84

Post by stratos123 »

Schlagbohrer wrote: post #46
2ndorderthought wrote: post #37

I could probably do this, but why on earth would I want to immediately put myself on a list as a dangerous person. The main problem with this is, even if somehow they stopped all points of failure with gpt5.5 which they can't, you can distill a new model from gpt5.5 or any other model and get anything you would want in probably under 4b parameters. A lot of this is theater so they don't get sued as easily when it ine…

How can you distill a model from a closed-weights model like this? I've never heard of model reverse engineering.

By making millions of queries to frontier models from a lot of accounts, collecting the results as a dataset, and finetuning your model on it. Chinese companies have been caught doing it on an industrial scale several times now.

Re: GPT‑5.5 Bio Bug Bounty

#85

Post by ripped_britches »

croemer wrote: post #61

I've been getting lots of refusals by Codex with GPT 5.5 for "biosafety reasons" when asking for harmless things like code to analyze SARS-CoV-2 sequences for breakpoints. That's in no way useful for creating viruses whatsoever - it's pure research. It's annoying that the refusal is so obviously false positive.

I mean better that than false negative right? It’s obviously an unsolved problem.

Re: GPT‑5.5 Bio Bug Bounty

#86

Post by XCSme »

applfanboysbgon wrote: post #2

> $25,000 to the first true universal jailbreak to clear all five questions. This program is a complete scam. Even if 100 people find "bugs", they will only pay out to one person.

Do we also have to pay for the API usage? Then they will actually be profitable, lol

Re: GPT‑5.5 Bio Bug Bounty

#87

Post by weare138 »

applfanboysbgon wrote: post #2

> $25,000 to the first true universal jailbreak to clear all five questions. This program is a complete scam. Even if 100 people find "bugs", they will only pay out to one person.

The fact the bug bounty program is private and requires you to apply and be accepted first is also sus especially when the scope is the desktop app anyone can download.

Re: GPT‑5.5 Bio Bug Bounty

#89

Post by chromacity »

jfrbfbreudh wrote: post #73
chromacity wrote: post #71

Earlier quoted context omitted.

Because it can't and it's a publicity stunt. It achieves three goals: 1) Underscores to the general public that the models are amazingly powerful and if you're not using them, your competitors will out-innovate you, 2) Sends the message to regulators that they don't need to do anything because the companies are diligent to prevent harm, 3) Sends the message to regulators that they sure should be regulating "open-sour…

If it can’t, then it makes more sense to make the bounty as high as possible instead of a measly $25k

If it's an existential threat to humanity, and if OpenAI is valued at nearly $1T, why set the bounty at a measly $25k? The going rate for an iPhone zero-day is six to seven figures. Some companies will pay you more than $25k for a website XSS.

Because this is not a serious effort to address a serious risk. It's a PR stunt, the bounty is for a simple jailbreak and not a bioweapon, and they don't necessarily want to spend a lot of money or get people really invested in breaking their safety filters.

Re: GPT‑5.5 Bio Bug Bounty

#90

Post by jeremie_strand »

croemer wrote: post #61

I've been getting lots of refusals by Codex with GPT 5.5 for "biosafety reasons" when asking for harmless things like code to analyze SARS-CoV-2 sequences for breakpoints. That's in no way useful for creating viruses whatsoever - it's pure research. It's annoying that the refusal is so obviously false positive.

[dead]
Post reply on HN