[dead]
Microsoft says bug causes Copilot to summarize confidential emails
71–80 of 85 posts
Re: Microsoft says bug causes Copilot to summarize confidential emails
#72The article doesn't say if the confidentiality labels were created with encryption. I've been using the latter (with Preview DLP) to prevent emails leaking out to _external_ integrations, which can't access the keys. With MS internal tooling, it's feasible that it access to the key, in which case that would be even worse. Does anyone know if this happened?
Re: Microsoft says bug causes Copilot to summarize confidential emails
#73Re: Microsoft says bug causes Copilot to summarize confidential emails
#74There are two issues I see here (besides the obvious “Why do we even let this happen in the first place?”): 1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone? 2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are curre…
I'm an AI researcher, here's my beliefs (it'll be clear in a second why I say belief and not claim objective facts)
1) you can't be sure it's gone. It's even questionable if data can be removed (longer discussion needed). These are compression machines, so the very act of training is compressing that information. The question really becomes how well that information is compressed or embedded into the model. On one hand, the models (typically) aren't invertible so the information is less likely to be compressed lodslessly. On the other hand, the models aren't invertible, so reversing them is probabilistic and they are harder to analyze in this sense.
2) as you may gather from 1) there's almost certainly more issues like this. There are many unknown unknowns waiting to be discovered. Personally this is why I'm very upset the field is so product focused and that a large portion regards theory as pointless. Theory does two things for us because it builds a deeper and more nuanced understanding. Theory advancing allows us to develop faster as we can iterate on paper rather than through experimentation. This allows us to better search the solution space and even understand our understanding. This also leads to better safety of models as it is necessary to understand them to understand where they fail and how to prevent those failures. Experimentation alone is incredibly naïve. It is like proving the correctness of your programs through testing (see the issues with TDD). Tests are great but they are bounds, not proofs. They can suggest safety, give you some level of confidence in safety, but they cannot guarantee it. We all know that the deeper understanding of your code the better tests you can write, and this is the same thing here. That theory is reducing your unknown unknowns and even before strong proofs are made we can get wider coverage in our testing.
I think we're so excited right now we're blinding ourselves. If we're cutting off or reducing fundamental research then we are killing the pipeline of development. Theory is the foundation that engineering sits on top of. But what worries me is that there's so many unknown unknowns and everyone is eagerly saying "we're just need 'good enough'" or "what's the minimum viable product". These are useful tools/questions but they have limits and it gets dangerous when putting out the minimum at scale
Re: Microsoft says bug causes Copilot to summarize confidential emails
#75Earlier quoted context omitted.
> All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. It depends. E.g. OpenAI says: "By default, we do not train on any inputs or outputs from our products for business users, including ChatGPT Team, ChatGPT Enterprise, and the API."[0] [0] https://openai.com/policies/how-your-data-is-used-to-improve...
"By default" is a fantastic escape catch in the language used there. So... What are the exceptions?
[deleted]
Re: Microsoft says bug causes Copilot to summarize confidential emails
#76Earlier quoted context omitted.
Yes, and those ideas are user hostile and poorly conceived, badly executed, and incompetently built. A remote code execution exploit in notepad ?! That's not professional, or skillful, or well done. Unnecessary feature bloat and change for the sake of change, because some MBA dork wants to justify their department and continued employment by checking boxes on spreadsheets. There's no innovation or skillful, well buil…
I agree except for Microsoft "failing". Windows is failing. Microsoft has moved onto other things.
azure is just as bad, if not worse
Re: Microsoft says bug causes Copilot to summarize confidential emails
#77Seems like every day there's another compelling reason to switch to Linux. Microsoft is doing truly incredible work this year!
The problem would still exist if you use Linux. This is a cloud service issue, not an OS issue.
Re: Microsoft says bug causes Copilot to summarize confidential emails
#78This company is an absolute joke now, if you're not desperately trying to jump ship at this point then you will go down with it.
Re: Microsoft says bug causes Copilot to summarize confidential emails
#79Just one more reason to abandon Microsoft. If ever Linux had an opportunity to breakout on the desktop, the proliferation of "AI" and privacy intrusion from the likes of Microsoft would seem to have opened that window. Yes, it would mean giving up some applications, at least temporarily, but the benefit in control and privacy makes that a fair trade off. Unlike the majority here, I don't want ANY "AI" features on my desktop, phone, car, or any appliance that I own. This is true of the "cloud" as well. Trusting corporate entities to have your best interests in mind is naive at best.
Re: Microsoft says bug causes Copilot to summarize confidential emails
#80Earlier quoted context omitted.
All the vendors paraphrase user data, then use the paraphrased data for training. This is what their terms of service say. They have significant experience in this. Microsoft software since the 2014, for the most part, is also paraphrased from other people's code they find laying around online.
Why would they want to train on random garbage proprietary emails? If their models ever spit out obviously confidential information belonging to their paying customers they'll lose those paying customers to their competitors - and probably face significant legal costs as well. Your random confidential corporate email really isn't that valuable for training. I'd argue it's more like toxic waste that should be avoided…
Your opinion seems a little unimaginative. To me, since email is the primary work output of millions of Americans, including all of its leaders, there is a lot of opportunity there.