Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

211–220 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#211

Post by DannyBee »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

"would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale"

Citation needed.

Most enterprise (homegrown or not) search engine products have to do this, and have been able to do it effectively at scale, for decades at this point.

This is a very well known and well-solved problem, and the solutions are very directly applicable to the products you list.

It is, as they say, a simple matter of implementation - if they don't offer it, it's because they haven't had the engineering time and/or customer need to do it.

Not because it doesn't scale.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#212

Post by malfist »

DannyBee wrote: post #211
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

"would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale" Citation needed. Most enterprise (homegrown or not) search engine products have to do this, and have been able to do it effectively at scale, for decades at this point. This is a very well known and well-solved problem, and the solutions are very directly applicable to the products you lis…

If you're stringing together a bunch of MCPs you probably also have to string together a bunch of authorization mechanisms. Try having your search engine confirm live each persons access to each possible row.

It's absolutely a hard problem and it isn't well solved

Re: Copilot broke audit logs, but Microsoft won't tell customers

#213

Post by nerdjon »

I am very curious realistically how can they reliably fix this.

So my understanding is that this is that the database/index that copilot used already crawled this file so of course it would not need to access the file to be able to tell the information in it.

But then, how do you fix that? Do you then tie audit reports to accessing parts of the database directly? Or are we instructing the LLM to do something like...

"If you are accessing knowledge pinky promise you are going to report it so we can add an audit log"

This really needs some communication from Microsoft on exactly what happened here and how it is being addressed since as of right now this should raise alarm bells for any company using Copilot and people have access to sensitive data that needs to be strictly monitored.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#214

Post by smarx007 »

PunchyHamster wrote: post #203
smarx007 wrote: post #200

Earlier quoted context omitted.

Two points/questions: 1. Why is tracking access rights "on a per user basis or [...] along with the content" is not feasible? A few mentions: Google Zanzibar (+Ory Keto as OSS impl) - makes authz for content othoronal to apps (i.e. possible to have it in one place, s.t. both Jira and a Jira MCP server can use the same API to check authz - possible to have a 100% faithful authz logic in the MCP server), Eclipse Biscui…

Because the problem is not "get a list of what user can access" but "the AI that got trained on dataset must not leak to user that doesn't have access to it. There is no feasible way to track that during training (at least yet), so only current solution would be to learn AI agent only on data use can access and that is costly

Who said it must be done during training? Most of the enterprise data is accessed after training - RAG or MCP tool calls. I can see how the techniques I mentioned above could be applied during RAG (in vector stores adopting Apache Accumulo ideas) or in MCP servers (MCP OAuth + RFC 8693 OAuth 2.0 Token Exchange + Zanzibar/Biscuit for faithfully replicating the authz constraints of systems where the data is being retrieved from).

Re: Copilot broke audit logs, but Microsoft won't tell customers

#215

Post by kuschku »

immibis wrote: post #185
fulafel wrote: post #79

> CVEs are given to fixes deployed in security releases when customers need to take action to stay protected. In this case, the mitigation will be automatically pushed to Copilot, where users do not need to manually update the product and a CVE will not be assigned. Is this a feature of CVE or of Microsoft's way of using CVE? It would seem this vulnerability would still benefit from having a common ID to be refrenced…

It's a feature of CVE. The C stands for Common.

I'd argue it'd still make sense to assign a CVE here. While you don't need to coordinate patching, many companies will need to issue reports to hipaa/gdpr oversight agencies, customers, employees, etc and having a common id for this vulnerability would make it easier to reference it and any related information.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#216

Post by smarx007 »

ForHackernews wrote: post #202
smarx007 wrote: post #200

Earlier quoted context omitted.

Two points/questions: 1. Why is tracking access rights "on a per user basis or [...] along with the content" is not feasible? A few mentions: Google Zanzibar (+Ory Keto as OSS impl) - makes authz for content othoronal to apps (i.e. possible to have it in one place, s.t. both Jira and a Jira MCP server can use the same API to check authz - possible to have a 100% faithful authz logic in the MCP server), Eclipse Biscui…

As I understand it, there's no real way to enforce access rights inside an LLM. If the bot has access to some data, and you have access to the bot, you can potentially trick it into coughing up the data regardless of whether you're supposed to see that info or not.

MCP tools with OAuth support + RFC 8693 OAuth 2.0 Token Exchange (aka OAuth 2.0 On-Behalf-Of flow in Azure Entra - though I don't think MCP 2025-06-18 accounts for the RFC 8693) could be used to limit the MCP bot responses to what the current user is authorized to see.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#217

Post by beaviskhan »

thunky wrote: post #205
doomslice wrote: post #198

Earlier quoted context omitted.

Let's say you have 100000 documents in your index that match your query but only 10 of them the user has access to: A basic implementation will return the top, let's say 1000, documents and then do the more expensive access check on each of them. Most of the time, you've now eliminated all of your search results. Your search must be access aware to do a reasonable job of pre-filtering the content to documents the use…

> Let's say you have 100000 documents in your index that match your query If the docs were indexed by groups/roles and you had some form of RBAC then this wouldn't happen.

If you take this approach, you have to reindex when groups/roles changes - not always a feasible choice

Re: Copilot broke audit logs, but Microsoft won't tell customers

#218

Post by verandaguy »

ath3nd wrote: post #37
verandaguy wrote: post #23

Earlier quoted context omitted.

I'm very sceptical of using shadow prompts (or prompts of any kind) as an actual security/compliance control or enforcement mechanism. These things should be done using a deterministic system.

I bet you are a fan of OpenAI's groundbreaking study mode feature.

I've heard of it by name, but not much beyond that.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#219

Post by smarx007 »

doomslice wrote: post #198
krisoft wrote: post #165

Earlier quoted context omitted.

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the pot…

Let's say you have 100000 documents in your index that match your query but only 10 of them the user has access to: A basic implementation will return the top, let's say 1000, documents and then do the more expensive access check on each of them. Most of the time, you've now eliminated all of your search results. Your search must be access aware to do a reasonable job of pre-filtering the content to documents the use…

Apache Accumulo solved the access-aware querying a while ago.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#220

Post by Isn0gud »

DannyBee wrote: post #207
doomslice wrote: post #198

Earlier quoted context omitted.

Let's say you have 100000 documents in your index that match your query but only 10 of them the user has access to: A basic implementation will return the top, let's say 1000, documents and then do the more expensive access check on each of them. Most of the time, you've now eliminated all of your search results. Your search must be access aware to do a reasonable job of pre-filtering the content to documents the use…

Yes. But this is still an incredibly well known and solved problem. As an example - google's internal structured search engines did this decades ago at scale.

Which solutions are you referring to? With access that is highly diverse and changing, this is still an unsolved problem to my knowledge.
Post reply on HN