Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI – vulnerability responsible disclosure

requilence.any.org

Post reply on HN
Print view View original

71–80 of 87 posts

Re: OpenAI – vulnerability responsible disclosure

#71

Post by BoiledCabbage »

requilence wrote: post #32
JyB wrote: post #15

Earlier quoted context omitted.

I don’t see anything here that would prevent a LLM from generating these. Right?

In one of the responses, it provided the financial analysis of a not well-known company with a non-Latin name located in a small country. I found this company; it is real and numbers in the response are real. When I asked my ChatGPT to provide a financial report for this company without using web tools, it responded: `Unfortunately, I don’t have specific financial statements for “xxx” for 2021 and 2022 in my training…

> numbers in the response are real.

OpenAI very well may have a bug, but I'm not clear on this part. How do you know the numbers are real?

I understand you know the name is the company is real, but how do you know the numbers are real?

It's way may than anyone should need to do, but the only way I can see someone knowing this is contacting the owners is the company.

Re: OpenAI – vulnerability responsible disclosure

#72

Post by [deleted] »

requilence wrote: post #2

Reported a flaw to OpenAI that lets users peek at others' chat responses. Got an auto-reply on May 29th, radio silence since. Issue remains unpatched :( Avoided their bug bounty due to permanent NDAs preventing disclosure even after fixes. Following standard 45-day disclosure window—users should avoid sharing sensitive data until this is resolved.

[deleted]

Re: OpenAI – vulnerability responsible disclosure

#73

Post by ragona »

diggan wrote: post #67
winstonhowes wrote: post #62

Hi all, I work on security at OpenAI. We have looked into this report and the model response does not contain outputs from any other users nor does it reflect a security vulnerability, compromise, or exploit. The original report was that submitting a message close to (but not quite) 1500 seconds to the audio transcription API would result in weird, unrelated, off-topic responses that look like they might be replies t…

> If you have found a security vulnerability, we encourage you to report it via our bug bounty program It seems like reporting bugs/issues via that program forces you to sign a permanent NDA preventing disclosures after the reported issue been fixed. I'm guessing the author of this disclosure isn't the only one that avoided it because of the NDA. Is that potentially something you can reconsider? Otherwise you'll prob…

(Note; I also work for OpenAI Security — though I’ve not worked on our bounty program for some time. These just my thoughts and experiences.)

I believe the author was referring to the standard BugCrowd terms, which as far as I know are themselves fairly common across the various platforms. In my experience we are happy for researchers to publish their work within the normal guidelines you’d expect from a bounty program — it’s something I’ve worked with researchers on without incident.

Re: OpenAI – vulnerability responsible disclosure

#74

Post by winstonhowes »

ragona wrote: post #73
diggan wrote: post #67

Earlier quoted context omitted.

> If you have found a security vulnerability, we encourage you to report it via our bug bounty program It seems like reporting bugs/issues via that program forces you to sign a permanent NDA preventing disclosures after the reported issue been fixed. I'm guessing the author of this disclosure isn't the only one that avoided it because of the NDA. Is that potentially something you can reconsider? Otherwise you'll prob…

(Note; I also work for OpenAI Security — though I’ve not worked on our bounty program for some time. These just my thoughts and experiences.) I believe the author was referring to the standard BugCrowd terms, which as far as I know are themselves fairly common across the various platforms. In my experience we are happy for researchers to publish their work within the normal guidelines you’d expect from a bounty progr…

100%. We want to ensure we can fix real security issues responsibly before details are published. In practice, if a researcher asks to disclose after we've addressed the issue, we're happy for them to publish.

Re: OpenAI – vulnerability responsible disclosure

#75

Post by DANmode »

addandsubtract wrote: post #68
requilence wrote: post #10

Earlier quoted context omitted.

No, definitely not the empty string hallucination bug. These are clearly real user conversations. They start like proper replies to requests, sometimes reference the original question, and appear in different languages.

New Touring Test unlocked! Differentiate between real and fake hallucinations.

So THAT'S what the "GT" means on all of these GPU model names!

Re: OpenAI – vulnerability responsible disclosure

#76

Post by DANmode »

winstonhowes wrote: post #74
ragona wrote: post #73

Earlier quoted context omitted.

(Note; I also work for OpenAI Security — though I’ve not worked on our bounty program for some time. These just my thoughts and experiences.) I believe the author was referring to the standard BugCrowd terms, which as far as I know are themselves fairly common across the various platforms. In my experience we are happy for researchers to publish their work within the normal guidelines you’d expect from a bounty progr…

100%. We want to ensure we can fix real security issues responsibly before details are published. In practice, if a researcher asks to disclose after we've addressed the issue, we're happy for them to publish.

In practice, it sounds like you guys didn't accept this dude's valid vuln because he didn't register and sign his life away.

Re: OpenAI – vulnerability responsible disclosure

#77

Post by JyB »

refulgentis wrote: post #70
queenkjuul wrote: post #66

Earlier quoted context omitted.

That the financial data is accurate?

It's an ourobos - he can't verify it's real! If he can, its online and available by search.

Therefore what are the odds that this is just the LLM doing its thing versus "a vulnerability". Seem like a pretty obvious bet.

Re: OpenAI – vulnerability responsible disclosure

#78

Post by JyB »

tabletcorry wrote: post #37
requilence wrote: post #16

Earlier quoted context omitted.

Right, thank you for the suggestion. Just added a paragraph to the original blog post.

Your added paragraph appears to suggest the opposite, that this was an LLM response. Was the "leaked data" a response from an LLM directly?

Yes apparently which makes this report pretty flimsy.

Re: OpenAI – vulnerability responsible disclosure

#79

Post by tptacek »

DANmode wrote: post #76
winstonhowes wrote: post #74

Earlier quoted context omitted.

100%. We want to ensure we can fix real security issues responsibly before details are published. In practice, if a researcher asks to disclose after we've addressed the issue, we're happy for them to publish.

In practice, it sounds like you guys didn't accept this dude's valid vuln because he didn't register and sign his life away.

They just stated it was all just model hallucination, and was not in fact a valid vuln.

Re: OpenAI – vulnerability responsible disclosure

#80

Post by tptacek »

JyB wrote: post #78
tabletcorry wrote: post #37

Earlier quoted context omitted.

Your added paragraph appears to suggest the opposite, that this was an LLM response. Was the "leaked data" a response from an LLM directly?

Yes apparently which makes this report pretty flimsy.

Upthread, OpenAI's security team confirms it's a false report; it's a variant of the empty-prompt hallucination.
Post reply on HN