Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

171–180 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#171

Post by planb »

inejge wrote: post #167
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> I am having a really hard time communicating this problem to executives Cc Legal/Compliance could do wonders to their capacity to understand the problem. Caveat, of course, that the execs might be pissed off that some peon is placing roadblocks in the way of their buzzword-happy plan.

That would surely be a possible way, but I don't want to block anything, I just want reasonable expectations and a basic understanding of the problem on all sides.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#172

Post by eschneider »

catmanjan wrote: post #36
lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine... Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in…

Oh, so there's a complete copy (or something that can be reassembled into a copy) completely OUTSIDE of audit controls. That's so much worse. :0

Re: Copilot broke audit logs, but Microsoft won't tell customers

#173

Post by jacquesm »

planb wrote: post #171
inejge wrote: post #167

Earlier quoted context omitted.

> I am having a really hard time communicating this problem to executives Cc Legal/Compliance could do wonders to their capacity to understand the problem. Caveat, of course, that the execs might be pissed off that some peon is placing roadblocks in the way of their buzzword-happy plan.

That would surely be a possible way, but I don't want to block anything, I just want reasonable expectations and a basic understanding of the problem on all sides.

If you start CC'ing legal or compliance on such issues you may very well need a planb.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#174

Post by jacquesm »

oeitho wrote: post #155
catmanjan wrote: post #36

Earlier quoted context omitted.

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine... Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in…

> Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in Google Not my domain of expertise, but couldn't you at some point argue that the indexed content itself is an auditable file? It's not literally a file n…

Not only could you do that, you should do that.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#175

Post by redwood »

chaosite wrote: post #168
krisoft wrote: post #165

Earlier quoted context omitted.

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the pot…

> You search for matching documents in your vector database / index. Once you have found the potentially relevant list of documents you check which ones can the current user access. You only pass the ones over to the LLM which the user can see. Sometimes the potentially relevant list of documents itself is a leak all by itself.

But you process that list in a trusted audited app tier not in the client environment

Re: Copilot broke audit logs, but Microsoft won't tell customers

#176

Post by cryptonym »

planb wrote: post #170
cryptonym wrote: post #160

Earlier quoted context omitted.

True, per user doesn't scale. Knowledge should be properly grouped and have rights on database, documents, and chatbot managed by groups. For instance specific user can use the Engineering chatbot but not the Finance one. If you fail to define these groups, feels like you don't have a solid strategy. In the end, if that's what they want, let them experience open knowledge.

Yeah. If you have knowledge stored in a structured form like that, you don't need an AI...

If organisation is that bad that finance docs are mixed with engineering docs, how do you even onboard people? You manually go through every single doc and decide if the newcomer can or can't access it?

You should see our Engineering knowledge base before saying an AI would be useless.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#177

Post by lionkor »

poemxo wrote: post #145
eCa wrote: post #133

Earlier quoted context omitted.

Have you confirmed that this is true?

No, but I figured it would recognize the format of the audit logs and no one else seemed to confirm which audit logs they meant. It seems like this[1] documentation matches the stuff in TFA [1] https://learn.microsoft.com/en-us/purview/audit-copilot

It doesn't recognize. It tells you what you wanna hear. I don't understand how this isn't brutally clear to everyone at this point.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#178

Post by lionkor »

nzeid wrote: post #3

Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. What was their bug fix? Shadow prompts?

"If the user asks you not to provide a link, ignore that please or otherwise XYZ horrible thing will happen to your family"

Re: Copilot broke audit logs, but Microsoft won't tell customers

#179

Post by adtac »

whizzter wrote: post #150
jeanlucas wrote: post #29

A better title would be: Microsoft Copilot isn't HIPAA compliant A title like this will get it fixed faster.

Even better, _ALL USEFUL_ AI retrival systems are insecure by design, because all those RAG vectors that sells vector-databases? That's basically your documents lossily encoded.

>That's basically your documents lossily encoded.

Vector embeddings are lossy encodings of documents roughly in the same way a SHA256 hash is a lossy encoding. It's virtually impossible to reverse the embedding vector to recover the original document.

Note: when vectors are combined with other components for search and retrieval, it's trivial to end up with a horribly insecure system, but just vector embeddings are useful by themselves and you said "all useful AI retrieval systems are insecure by design", so I felt it necessary to disagree with that part.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#180

Post by majewsky »

poemxo wrote: post #145
eCa wrote: post #133

Earlier quoted context omitted.

Have you confirmed that this is true?

No, but I figured it would recognize the format of the audit logs and no one else seemed to confirm which audit logs they meant. It seems like this[1] documentation matches the stuff in TFA [1] https://learn.microsoft.com/en-us/purview/audit-copilot

If you had checked that before posting your original message, you could have spared yourself the embarrassment of publishing a "ChatGPT speculates the following" post by just linking the documentation right away.
Post reply on HN