Live data from Hacker News

Employees are feeding sensitive data to ChatGPT, raising security fears

darkreading.com

131–140 of 355 posts

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#131
post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

Does blocking ever work? People are smart and usually just work around them.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#132
post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

This really depends on the cost/benefit tradeoff for the entity in question. If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage, is that worth it in expectation or not? I would argue that there definitely exist companies for which it's worth the tradeoff. By the way, OpenAI says they wont use data submitted through its API for model train…

To anyone who may be pasting code along the lines of 'convert this sql table schema into a [pydantic model|JSON Schema]' where you're pasting in the text, just ask it instead to write you a [python|go|bash|...] function that reads in a text file and 'converts an sql table schema to output x' or whatever. Related/not-related--great pandas docs replacement is another great+safe use-case.

Point is, for a meaningful subset of high-value use-cases you don't need to move your important private stuff across any trust boundaries, and it still can be pretty helpful...so just calling that out in case that's useful to anyone...

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#133
post #71

ChatGPT Business Edition seems pretty obvious and I'd surprised if OpenAI isn't already working on it. Separate models for each customer, data silos and protection. The infra is already there on Azure.

It actually is on Azure, exactly as you described. https://learn.microsoft.com/en-us/azure/cognitive-services/o...

Yep, they just need to provide a business specific frontend chat UI.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#134

We saw these same fears with the release of Gmail. Why would you trust your email to Google?!! Aren't they going to train their spam filters on all your data? Aren't they going to sell it, or use it to sell you ads? Corporations constantly put their most sensitive data in 3rd party tools. The executive in the article was probably copying his company strategy from Google docs. Yes, there are good reasons for concern,…

If your company's code is all repositories on Github (or bitbucket, or any similar service), worrying about ChatGPT is quite silly.

And on the other hand, if your company doens't use Github etc due to security concern, it's a very good sign telling you need to ban ChatGPT too.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#135
post #91

Earlier quoted context omitted.

But you created a throwaway account specifically to reply in this thread? Unless your company really has nothing to hide, it's easy to accidentally dump a company secret or an API key in a chat session. Of course if everyone is aware of this and constantly careful then you may be OK.

That's because accounts get shadow banned all the time when people get upset when you point out hard truths. If you're copy pasting API keys or such into ANYTHING, you probably shouldn't be a programmer to begin with. It's like people who use root account key/secret credentials in their codebase. It's not AWSs fault you got a large bill or got hacked, its because you're dumb.

I posted my openAI token into a GitHub issue today thinking I'd just kill it right away, which I did but there was already an email from openAI letting me know that it was noticed that my token had become public and was thus revoked.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#136
post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

Does blocking ever work? People are smart and usually just work around them.

The smart ones don’t paste in all their private data.

And yes, if the bypassing the block is combined with disciplinary action, it does work. It’s not worth getting fired over. This is likely what heavily regulated industries like financial services and defense are doing.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#137
just because your code is art of course, which should be weighted in gold and copyrighted for the next 2000 years living in a cold wallet. Did you get that memo, that humans implemented all the faulty security that lasted the past decades by accident?

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#138
post #5

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model - OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away…

Does blocking ever work? People are smart and usually just work around them.

Blocks are effective reminders of policies.

Re: Employees are feeding sensitive data to ChatGPT, raising security fears

#140
post #111

Earlier quoted context omitted.

Trusting Gmail with corporate communication was was a terrible idea (and explicitly illegal in a lot of industries), and companies didn't start to adopt it until Google released an enterprise version with table-stakes security features like no training on the data, no ad targeting, auditing, compliance holds and more. There's a huge difference between trusting a third party service with strict security and data priva…

Was?

Well it was, until Google Workspace (G Suite) came along and provided essentially an enterprise version of Gmail.
Post reply on HN