Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

181–190 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#181

Post by lukan »

krisoft wrote: post #165
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the pot…

Thank you for context, I wondered the same.

But I guess they want something like training the chatbot as a LLM once with all the confidential data - and then indeed you could never separate it again.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#182

Post by immibis »

fulafel wrote: post #110
immibis wrote: post #54

Earlier quoted context omitted.

More accurately, CVEs are for vulnerabilities that may be present on many systems. Then, the CVE number is a reference point that helps you when discussing the vulnerability, like asking whether it's present on a particular system, or what percentage of systems are patched. This vulnerability was only present on one system, so it doesn't need a CVE number. It could have a Microsoft-assigned bug number, but it doesn't…

This may be a stated reason but it's questionable logic. There are of course many cases where people need to reference and discuss this vulnerability and its impact.

There are many cases where people need to reference and discuss the weather, but the weather doesn't need a CVE number. If you could hypothetically put it in a known vulnerability scanner then it should have a CVE. Otherwise no.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#183

Post by lixtra »

redwood wrote: post #175
chaosite wrote: post #168

Earlier quoted context omitted.

> You search for matching documents in your vector database / index. Once you have found the potentially relevant list of documents you check which ones can the current user access. You only pass the ones over to the LLM which the user can see. Sometimes the potentially relevant list of documents itself is a leak all by itself.

But you process that list in a trusted audited app tier not in the client environment

A naive approach could still leak information through side channels. E.g. if you search regularly for foobar, the answer might suddenly get slower if foobar appears more in the document base.

Depending on the context it could be relevant.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#184

Post by Drakim »

eschneider wrote: post #172
catmanjan wrote: post #36

Earlier quoted context omitted.

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine... Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in…

Oh, so there's a complete copy (or something that can be reassembled into a copy) completely OUTSIDE of audit controls. That's so much worse. :0

It's roughly the same problem as letting a search engine build indexes (with previews!) of sites without authentication. It's kinda crazy that things were allowed to go this far with such a fundamental flaw.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#185

Post by immibis »

fulafel wrote: post #79

> CVEs are given to fixes deployed in security releases when customers need to take action to stay protected. In this case, the mitigation will be automatically pushed to Copilot, where users do not need to manually update the product and a CVE will not be assigned. Is this a feature of CVE or of Microsoft's way of using CVE? It would seem this vulnerability would still benefit from having a common ID to be refrenced…

It's a feature of CVE. The C stands for Common.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#186

Post by mr_toad »

pnt12 wrote: post #156
RachelF wrote: post #84

Earlier quoted context omitted.

Lots of things aren't clear. Copilot is accessing the indexed contents of the file, not the file itself, when you tell it not to access the file. The blog writer/marketer needs to look at the index access logs.

But those are technicalities. I imagine the intended feature is learning about who read some information, and who modified it. The implementation varies, but on a CRUD app it seems easy: an authenticated GET or PUT request against a file path - easy audit log. If you are copying information to another place, and make it accessible there in a lossy way that is hard to audit... you broke your auditing system. Maybe it'…

Not all vector databases are lossy. And even if a lossy index is used it’s totally possible to identify the original source of the information.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#187

Post by swiftcoder »

bw86 wrote: post #151
internetter wrote: post #86

Earlier quoted context omitted.

> I don't say I've visited a website when I've found a result of it in Google I mean, it depends on how large the index window is, because if google returned the entire webpage content without leaving (amp moment), you did visit the website. fine line.

The challenge then is to differentiate between "I wanted to access the secret website/document" and "Google/Copilot gave me the secret website/document, but it was not my intention to access that".

Access is access. Regardless of whether you intended to view the document, you are now aware of its content in either case, and an audit entry must be logged.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#188

Post by swiftcoder »

gpm wrote: post #40
Spooky23 wrote: post #28

Earlier quoted context omitted.

No, it accesses data with the users privilege.

Are you telling me I, a normal unprivileged user, have a way to read files on windows that bypasses audit logs?

If someone (Copilot, in this case) has built a search index that covers all the files on your computer, and left it accessible to your user account... yes

Re: Copilot broke audit logs, but Microsoft won't tell customers

#189

Post by [deleted] »

adtac wrote: post #179
whizzter wrote: post #150

Earlier quoted context omitted.

Even better, _ALL USEFUL_ AI retrival systems are insecure by design, because all those RAG vectors that sells vector-databases? That's basically your documents lossily encoded.

>That's basically your documents lossily encoded. Vector embeddings are lossy encodings of documents roughly in the same way a SHA256 hash is a lossy encoding. It's virtually impossible to reverse the embedding vector to recover the original document. Note: when vectors are combined with other components for search and retrieval, it's trivial to end up with a horribly insecure system, but just vector embeddings are u…

[deleted]
Post reply on HN