Live data from Hacker News

Microsoft says bug causes Copilot to summarize confidential emails

bleepingcomputer.com

51–60 of 85 posts

Re: Microsoft says bug causes Copilot to summarize confidential emails

#51

Seems like every day there's another compelling reason to switch to Linux. Microsoft is doing truly incredible work this year!

I recently switched my work laptop from a Dell to a MacBook. I found out that windows 11 has so much corporate bloat, than even MS apps like outlook, office and OneDrive functions better on a Mac than on Windows 11.

Re: Microsoft says bug causes Copilot to summarize confidential emails

#52

Earlier quoted context omitted.

> "The Microsoft 365 Copilot 'work tab' Chat is summarizing email messages even though these email messages have a sensitivity label applied and a DLP policy is configured." I'd add (3) - a DLP policy is apparently ineffective at its purpose: monitoring data sharing between machines. ( https://learn.microsoft.com/en-us/purview/dlp-learn-about-dl... ). Directly from the DLP feature page: > DLP, with collection policie…

> a DLP policy is apparently ineffective at its purpose /Offtopic Yes, MSFT's DLP/software malfunctioned, but getting users to MANUALLY classify things as confidential is already an uphill battle. These are for the rare subset of people that are aware of and compliant with NDAs/Confidentiality Agreements!

Who can blame them, when in the end, it gets ignored anyways?

Re: Microsoft says bug causes Copilot to summarize confidential emails

#53

Seems like every day there's another compelling reason to switch to Linux. Microsoft is doing truly incredible work this year!

Apple not doing much better, but from the other end. Microsoft releasing overly ambitious features with disastrous consequences. Apple releasing features so unambitious it's hard to remember they're there.

The problem with the Microsoft features is really not excessive ambition.

Half of the time it's open user hostility and blatant incompetence. The other half it's just the incompetence. Ambition doesn't enter the picture at all.

Re: Microsoft says bug causes Copilot to summarize confidential emails

#56

Earlier quoted context omitted.

All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. They have significant experience in this. Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online.

> Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online. That was pretty funny and explains a lot. I wish I could do more :( Instead I always break things when I paraphrase code without the GeniusParaphrasingTool

This is exactly why I moved to self hosted code in 2017.

While I couldn’t have predicted the future, even classic data mining posed a risk.

It is just reality that if you give a third party access to your data, you should expect them to use it.

It is just too tempting of a value stream and legislation just isn’t there to avoid the EULA trap.

I was targeting a market where fractions of a percentage advantage were important which did drive my what at the time was labeled paranoia

Re: Microsoft says bug causes Copilot to summarize confidential emails

#57
post #20

There are two issues I see here (besides the obvious “Why do we even let this happen in the first place?”): 1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone? 2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are curre…

Copilot is not a model, to my knowledge. When you’re asking about the data that it was trained on, you are most likely referring to an OpenAI or, in some circumstances, an Anthropic model. Customer data is not used for training the models that run Copilot.

Re: Microsoft says bug causes Copilot to summarize confidential emails

#58
post #20

There are two issues I see here (besides the obvious “Why do we even let this happen in the first place?”): 1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone? 2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are curre…

All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. They have significant experience in this. Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online.

Why would they want to train on random garbage proprietary emails?

If their models ever spit out obviously confidential information belonging to their paying customers they'll lose those paying customers to their competitors - and probably face significant legal costs as well.

Your random confidential corporate email really isn't that valuable for training. I'd argue it's more like toxic waste that should be avoided at all costs.

Re: Microsoft says bug causes Copilot to summarize confidential emails

#59

Earlier quoted context omitted.

All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. They have significant experience in this. Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online.

> All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. It depends. E.g. OpenAI says: "By default, we do not train on any inputs or outputs from our products for business users, including ChatGPT Team, ChatGPT Enterprise, and the API."[0] [0] https://openai.com/policies/how-your-data-is-used-to-improve...

"By default" is a fantastic escape catch in the language used there. So... What are the exceptions?
Post reply on HN