I'm curious which mailbox they sent to, trying to find a mailbox is surprisingly hard even with my Google searching.
they have security.txt file on their domain and mentioned it in some other place
OpenAI – vulnerability responsible disclosure
21–30 of 87 posts
Re: OpenAI – vulnerability responsible disclosure
#22> The leaked responses show clear signs of being real conversations: they start with contextually appropriate replies, sometimes reference the original user question, appear in various languages, and maintain coherent conversational flow. This pattern is inconsistent with random model hallucinations but matches exactly what you'd expect from misdirected user sessions.
A model like GPT-4o can hallucinated responses that are indistinguishable from real user interactions. This is easy to confirm for yourself: just ask it to make one up.
I’m certainly willing to believe OpenAI leaks real user messages, but this is not proof of that claim.
Re: OpenAI – vulnerability responsible disclosure
#23[deleted]
Re: OpenAI – vulnerability responsible disclosure
#24Reported a flaw to OpenAI that lets users peek at others' chat responses. Got an auto-reply on May 29th, radio silence since. Issue remains unpatched :( Avoided their bug bounty due to permanent NDAs preventing disclosure even after fixes. Following standard 45-day disclosure window—users should avoid sharing sensitive data until this is resolved.
Permanent NDA's? Oof. It's like their plan is to just try to force the lid down till they reach ASI or something lol
Again: NDAs are bog standard bounty terms.
Re: OpenAI – vulnerability responsible disclosure
#25Earlier quoted context omitted.
The NDA part feels really murky.
It's pretty standard for bounty programs. If you don't like it, which is reasonable, do what this researcher did and just post independently.
The bug bounty world is a funny one. I remember one complaining that their bug was dismissed and fixed after they signed an NDA, no payout, nothing. Another one got $100 instead of $5,000 because the company downgraded the severity from high to low. So they ended up with little or no money, and no recognition either. Not sure if these were edge cases, but it does make you wonder how fair the process really is.
Re: OpenAI – vulnerability responsible disclosure
#26> The leaked responses show clear signs of being real conversations: they start with contextually appropriate replies, sometimes reference the original user question, appear in various languages, and maintain coherent conversational flow. This pattern is inconsistent with random model hallucinations but matches exactly what you'd expect from misdirected user sessions. A model like GPT-4o can hallucinated responses th…
Ya, hard to know how to react without more information.
Re: OpenAI – vulnerability responsible disclosure
#27Earlier quoted context omitted.
It's pretty standard for bounty programs. If you don't like it, which is reasonable, do what this researcher did and just post independently.
The bug bounty world is a funny one. I remember one complaining that their bug was dismissed and fixed after they signed an NDA, no payout, nothing. Another one got $100 instead of $5,000 because the company downgraded the severity from high to low. So they ended up with little or no money, and no recognition either. Not sure if these were edge cases, but it does make you wonder how fair the process really is.
If you're dealing with large companies, a good rule of thumb is that the bounty program is incentivized to pay you out. Their internal metrics improve the more they pay; the point is to turn up interesting bugs, and the figure of merit for that is "how much did we have to spend". At a large company, a bounty that isn't paying anything out is a failure.
All bets are off with small random startups that do bug bounties because they think they're supposed to (most companies should not run bounties). But that's not OpenAI. Dave Aitel works at OpenAI. They're not trying to stiff you.
Simultaneous discovery (either with other researchers or, even more often, with internal assessments) is super common. What's more, you're not going to get any corroboration or context for them (sets up a crazy bad incentive with bounty seekers, who litigate bounty results endlessly). When you get a weird and unfair-seeming response to a bounty from a big tech company, for the sake of your own sanity (and because you'll probably be right), just assume someone internal found the bug before you did, and you reported it in the (sometimes long) window during which they were fixing it.
Re: OpenAI – vulnerability responsible disclosure
#28Earlier quoted context omitted.
you're sure it's not their "feature" that calling the api with empty string returns random hallucinations? https://jarbon.medium.com/gpt-prompt-bug-94322a96c574
No, definitely not the empty string hallucination bug. These are clearly real user conversations. They start like proper replies to requests, sometimes reference the original question, and appear in different languages.
i had the exact same behavior back in 2023, it seemed like clearly leakage of user conversations - but it was just a bug with api calls in the software i was using.
Re: OpenAI – vulnerability responsible disclosure
#29Earlier quoted context omitted.
The NDA part feels really murky.
It's pretty standard for bounty programs. If you don't like it, which is reasonable, do what this researcher did and just post independently.
That's an exaggeration. Most industry leaders do not require NDAs, only coordinated disclosure.
Mozilla's program, which has been around longer than most, doesn't. Google and Microsoft don't. Meta and Apple don't.
This is water carrying, intentional or not, for a terrible practice that should be shamed, so that it doesn't become standard.
Re: OpenAI – vulnerability responsible disclosure
#30> The leaked responses show clear signs of being real conversations: they start with contextually appropriate replies, sometimes reference the original user question, appear in various languages, and maintain coherent conversational flow. This pattern is inconsistent with random model hallucinations but matches exactly what you'd expect from misdirected user sessions. A model like GPT-4o can hallucinated responses th…
In one of the responses, it provided the financial analysis of a not well-known company with a non-Latin name located in a small country. I found this company; it is real and numbers in the response are real. When I asked my ChatGPT to provide a financial report for this company without using web tools, it responded: `Unfortunately, I don’t have specific financial statements for “xxx” for 2021 and 2022 in my training data, and since you’ve asked not to use web search, I can’t pull them live.`.