Live data from Hacker News

OpenAI – vulnerability responsible disclosure

requilence.any.org

81–87 of 87 posts

Re: OpenAI – vulnerability responsible disclosure

#81
post #79
post #76

Earlier quoted context omitted.

In practice, it sounds like you guys didn't accept this dude's valid vuln because he didn't register and sign his life away.

They just stated it was all just model hallucination, and was not in fact a valid vuln.

shrugs If you're convinced, I'm convinced!

Re: OpenAI – vulnerability responsible disclosure

#83
post #80
post #78

Earlier quoted context omitted.

Yes apparently which makes this report pretty flimsy.

Upthread, OpenAI's security team confirms it's a false report; it's a variant of the empty-prompt hallucination.

Incredible that so many people still don't understand what an LLM is. Especially ones that you would expect to grasp it.

Re: OpenAI – vulnerability responsible disclosure

#84

Hi all, I work on security at OpenAI. We have looked into this report and the model response does not contain outputs from any other users nor does it reflect a security vulnerability, compromise, or exploit. The original report was that submitting a message close to (but not quite) 1500 seconds to the audio transcription API would result in weird, unrelated, off-topic responses that look like they might be replies t…

Thank you, I made an update to the original post with your explanation, and because you stated that the output was a pure hallucination, I also attached one of them.

Re: OpenAI – vulnerability responsible disclosure

#85

Earlier quoted context omitted.

In one of the responses, it provided the financial analysis of a not well-known company with a non-Latin name located in a small country. I found this company; it is real and numbers in the response are real. When I asked my ChatGPT to provide a financial report for this company without using web tools, it responded: `Unfortunately, I don’t have specific financial statements for “xxx” for 2021 and 2022 in my training…

Did you try to ask it to provide data of the company, by explicitly invoking hallucination in the model? Right now there is no real proof, untill you confirm that the data it provided cannot be hallucinated (which could be not feisable). Also, acknowledging the response fron OpenAI staff dismissing it, would you mind sharing PoC?

I've updated the original post with technical details and an output example.

Re: OpenAI – vulnerability responsible disclosure

#86
post #31

Earlier quoted context omitted.

That's an exaggeration. Most industry leaders do not require NDAs, only coordinated disclosure. Mozilla's program, which has been around longer than most, doesn't. Google and Microsoft don't. Meta and Apple don't. This is water carrying, intentional or not, for a terrible practice that should be shamed, so that it doesn't become standard.

My understanding is that all Bugcrowd bounties do by default. You can shame it all you want, but you can also just publish your bugs directly. Nobody has to use the Bugcrowd platform. You don't even have to wait 45 days; I don't buy these "CERT/CC" rules.

You said it was pretty standard for bug bounty programs, and I disagreed pointing to several of the largest and longest lived bug bounty programs, none of which do that, and your response is pointing out that one particular platform does it?

Even among 3rd party platforms, of which there are several bigs, the NDAs are not a platform requirement, just an option for participating firms.

NDAs are not the norm. Don't mislead people who would otherwise get into this game with non-issues they need not worry over.

Re: OpenAI – vulnerability responsible disclosure

#87
post #31

Earlier quoted context omitted.

My understanding is that all Bugcrowd bounties do by default. You can shame it all you want, but you can also just publish your bugs directly. Nobody has to use the Bugcrowd platform. You don't even have to wait 45 days; I don't buy these "CERT/CC" rules.

You said it was pretty standard for bug bounty programs, and I disagreed pointing to several of the largest and longest lived bug bounty programs, none of which do that, and your response is pointing out that one particular platform does it? Even among 3rd party platforms, of which there are several bigs, the NDAs are not a platform requirement, just an option for participating firms. NDAs are not the norm. Don't mis…

OpenAI's security team commented on the thread themselves that they believe they simply accepted the Bugcrowd defaults. I think you're trying to find a controversy that just isn't here.
Post reply on HN