Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Employees are feeding sensitive data to ChatGPT, raising security fears

darkreading.com

Post reply on HN
Print view View original

201–210 of 355 posts

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#201

Post by nomilk »

cuuupid wrote: post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

> there’s no way they’re manually scrubbing out sensitive data

I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#202

Post by mewpmewp2 »

chrismorgan wrote: post #198
beiller wrote: post #143

Earlier quoted context omitted.

I did not verify this regex, on the very surface scan seems OK: ./main -m ./models/7B/ggml-model-q4_0.bin -t 4 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647 -n 1024 -p $'Here is a handy short form regex to validate an email address: ' Here is a handy short form regex to validate an email address: ^([a-zA-Z0-9_.+-]+@[a-zA-Z0-9.-]+\.[a-z\.]{2,6})+$ The first character can be either uppe…

Badly wrong, in part because your prompt was badly wrong, because there is no short regex to validate an email address. (It needs a way of saying “I can’t complete that because the prompt makes no sense for this reason”.) What it emitted accepts a large number of invalid addresses (due to things like not checking dot placement, and the inexplicable (…)+ wrapping around the entire thing), and doesn’t accept a large nu…

It is a bit crazy to me someone posts a regex like that without verifying and saying on surface level it looks good, implying the whole thing was useful and a good result.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#203

Post by belorn »

wantsanagent wrote: post #78

We saw these same fears with the release of Gmail. Why would you trust your email to Google?!! Aren't they going to train their spam filters on all your data? Aren't they going to sell it, or use it to sell you ads? Corporations constantly put their most sensitive data in 3rd party tools. The executive in the article was probably copying his company strategy from Google docs. Yes, there are good reasons for concern,…

If a company/government risk model allows for giving Google all the most sensitive information, then that says something about trust. It has not gone unnoticed how those risk model differs when something like tiktok arrived, or with earlier Huawei 5G modems.

The enterprise version of gmail was just an additional step to instill trust. In practice it is still a decision based on trust rather than physics. An "enterprise version of privacy guaranties" for Huawei 5G modems or tiktok apps would not make governments suddenly happy with the risk model where sensitive data would have a minor risk of ending up in China.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#204

Post by whitemary »

nomilk wrote: post #201
cuuupid wrote: post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

> there’s no way they’re manually scrubbing out sensitive data I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.

Even if they do today, they could change their mind tomorrow, or even tonight.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#205

Post by syntaxing »

AnthonyMouse wrote: post #194
syntaxing wrote: post #22

I’m curious if anyone’s employer has set up their own LLM. My employer has a couple of A100 sitting around which could easily host a couple instance of 65B LLaMA or Alpaca. Convincing upper management to allow me is the hard part.

I just run it on my desktop? 64GB of DDR4 is <$150.

I'm assuming a quantized version?

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#206

Post by a_bonobo »

knodi123 wrote: post #44
meghan_rain wrote: post #13

Earlier quoted context omitted.

Inform us when you figured out a way to host something with the quality of ChatGPT internally :-)

I can host llama's 7b model internally. It hallucinates more often than not and has a tendency to ramble, but dammit it's local and secure!

llama's 7b model internally is, for me, on a totally different level quality-wise. Even when explicitly instructed to not make up stuff and just say 'I don't know', it will still go ahead and ramble and invent things. When I tell it to only use the prompt data it will still invent, or just ignore the prompt data. It's not useful for production (i.e., to be exposed to 'regular' non AI users). ChatGPT, on the other hand, will listen to those instructions and say when it does not know, and will keep to only the prompt data.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#207

Post by gwright »

ChatPGT wrote: post #164
osterbit2 wrote: post #132

Earlier quoted context omitted.

To anyone who may be pasting code along the lines of 'convert this sql table schema into a [pydantic model|JSON Schema]' where you're pasting in the text, just ask it instead to write you a [python|go|bash|...] function that reads in a text file and 'converts an sql table schema to output x' or whatever. Related/not-related--great pandas docs replacement is another great+safe use-case. Point is, for a meaningful subs…

I'm doing that since day one. I can't believe people are pasting real data into this corporate black boxes.

What about Google Docs, Office 365, Github, AWS, Azure, Google Cloud, JIRA, Zendesk, etc?

What is different about ChatGPT (if anything)?

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#209

Post by Schroedingersat »

nomilk wrote: post #201
cuuupid wrote: post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

> there’s no way they’re manually scrubbing out sensitive data I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.

Right. They also haven't been caught lying about everything else from cuttoff dates to live functionality.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#210

Post by mc32 »

Teever wrote: post #196
ChatPGT wrote: post #164

Earlier quoted context omitted.

I'm doing that since day one. I can't believe people are pasting real data into this corporate black boxes.

To your average user who interfaces with these figurative black boxes with a black box in their hand, how is this particular black box any different than the other black boxes that this user hands their data to every second of every day?

Those typically have MSAs with legalese where parties stipulate what they will and will not do and often whether or not it’s zero knowledge and often option to have your own instance encryption keys.

If people are using the free version of chatGPT then it’s unlikely there is a contract between the companies and more likely just a terms of use applied by chatGPT and ignored by the users.

Post reply on HN