Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI – vulnerability responsible disclosure

requilence.any.org

Post reply on HN
Print view View original

11–20 of 87 posts

Re: OpenAI – vulnerability responsible disclosure

#11

Post by 999900000999 »

requilence wrote: post #2

Reported a flaw to OpenAI that lets users peek at others' chat responses. Got an auto-reply on May 29th, radio silence since. Issue remains unpatched :( Avoided their bug bounty due to permanent NDAs preventing disclosure even after fixes. Following standard 45-day disclosure window—users should avoid sharing sensitive data until this is resolved.

Users should always avoid sharing sensitive data.

A lot of AI products straight up have plan text logs available for everyone at the company to view.

Re: OpenAI – vulnerability responsible disclosure

#12

Post by JyB »

I believe it is extremely important to disclose that the ‘responses leaks’ you obtained did not originate from LLM models themselves, but rather through other insecure systems / in a more conventional manner.

Just to avoid yet another case of hallucinations outputs getting misinterpreted.

Re: OpenAI – vulnerability responsible disclosure

#13

Post by pyman »

> A single misconfiguration can leak thousands of sensitive conversations in seconds. Treating privacy as an afterthought is untenable when the blast radius is this large.

Massive security bug, well spotted. It's like Bank of America showing other people my transactions, or Meta leaking my WhatsApp messages.

This raises some serious questions about security.

Re: OpenAI – vulnerability responsible disclosure

#14

Post by ameliaquining »

999900000999 wrote: post #11
requilence wrote: post #2

Reported a flaw to OpenAI that lets users peek at others' chat responses. Got an auto-reply on May 29th, radio silence since. Issue remains unpatched :( Avoided their bug bounty due to permanent NDAs preventing disclosure even after fixes. Following standard 45-day disclosure window—users should avoid sharing sensitive data until this is resolved.

Users should always avoid sharing sensitive data. A lot of AI products straight up have plan text logs available for everyone at the company to view.

Which ones? Do you just mean tiny startups and side projects and the like or is this a problem that major model providers have?

Re: OpenAI – vulnerability responsible disclosure

#15

Post by JyB »

requilence wrote: post #10
jonrouach wrote: post #6

Earlier quoted context omitted.

you're sure it's not their "feature" that calling the api with empty string returns random hallucinations? https://jarbon.medium.com/gpt-prompt-bug-94322a96c574

No, definitely not the empty string hallucination bug. These are clearly real user conversations. They start like proper replies to requests, sometimes reference the original question, and appear in different languages.

I don’t see anything here that would prevent a LLM from generating these. Right?

Re: OpenAI – vulnerability responsible disclosure

#16

Post by requilence »

JyB wrote: post #12

I believe it is extremely important to disclose that the ‘responses leaks’ you obtained did not originate from LLM models themselves, but rather through other insecure systems / in a more conventional manner. Just to avoid yet another case of hallucinations outputs getting misinterpreted.

Right, thank you for the suggestion. Just added a paragraph to the original blog post.

Re: OpenAI – vulnerability responsible disclosure

#18

Post by blibble »

good to see more and more hackers refusing to use corporate bug bounty platforms with onerous terms

I certainly wouldn't sign an indefinite NDA for a chance to win:

Average payout: $836.36

openai should be grateful, after all, they want all information to be free

Re: OpenAI – vulnerability responsible disclosure

#20

Post by pyman »

999900000999 wrote: post #11
requilence wrote: post #2

Reported a flaw to OpenAI that lets users peek at others' chat responses. Got an auto-reply on May 29th, radio silence since. Issue remains unpatched :( Avoided their bug bounty due to permanent NDAs preventing disclosure even after fixes. Following standard 45-day disclosure window—users should avoid sharing sensitive data until this is resolved.

Users should always avoid sharing sensitive data. A lot of AI products straight up have plan text logs available for everyone at the company to view.

It's not just about sensitive data like passwords, contracts, or IP. It's also about the personal conversations people have with ChatGPT. Some are depressed, some are dealing with bullying, others are trying to figure out how to come out to their parents. For them, this isn't just sensitive, it's life-changing if it gets leaked. It's like Meta leaking their WhatsApp messages.

I really hope they fix this bug and start taking security more seriously. Trust is everything.

Post reply on HN