Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

31–40 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#31

Post by nzeid »

nzeid wrote: post #27
jsnell wrote: post #14

Earlier quoted context omitted.

> Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. Nothing in this post suggests that they're relying on the LLM itself to append to the audit logs. That would be a preposterous design. It seems far more likely the audit logs are being written by the scaffolding, not by the LLM, but they instrumented the wrong places. (I.e. emitting on a link or ma…

Heard, but since the content or its metadata must be surfaced by the LLM, what's the fix?

Thinking about this a bit - you'd have to isolate any interaction the LLM has with any content to some sort of middle end that can audit the LLM itself. I'm a bit out of my depth here, though. I don't know what Microsoft does or doesn't do with Copilot.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#32

Post by crooked-v »

bongodongobob wrote: post #17
Josh5 wrote: post #13

are they even sure that the AI even accessed the content that second time? LLMs are really good and making up shit. I have tested this by asking various LLMs to scrape data from my websites while watching access logs. Many times, they don't and just rely on some sort of existing data or spout a bunch of BS. Gemini is especially bad like this. I have not used copilot myself, but my experience with other AI makes me cu…

This is it. M365 uses RAG on your enterprise data that you allow it to access. It's not actually accessing the files directly in the cases he provided. It's working as intended.

If that's the case, then as noted in the article, the 'as intended' is probably violating liability requirements around various things.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#33

Post by Spooky23 »

QuadmasterXLII wrote: post #18

This seems like a five alarm fire for HIPPA, is there something I’m missing?

It’s a bug. He reported it, they fixed it.

It is not a five alarm fire for HIPAA. HIPAA doesn’t require that all file access be logged at all. HIPAA also doesn’t require that a CVE be created for each defect in a product.

End of the day, it’s a hand-wavy, “look at me” security blog. Don’t get too crazy.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#34

Post by userbinator »

heywire wrote: post #9

I am so tired of Microsoft cramming Copilot into everything. Search at $dayjob is completely borked right now. It shows a page of results, but the immediately pops up some warning dialog you cannot dismiss that Copilot can’t access some file “” or something. Every VSCode update I feel like I have to turn off Copilot in some new way. And now apparently it’ll be added to Excel as well. Thankfully I don’t have to use an…

Thankfully I don’t have to use anything from Microsoft after work hours.

There are employers where you don't have to use anything from Microsoft during work hours either.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#35

Post by jjkaczor »

lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

So... basically like when Delve was first introduced and was improperly security trimming things it was suggesting and search results.

... Or ... a very long-time ago, when SharePoint search would display results and synopsis's for search terms where a user couldn't open the document, but could see that it existed and could get a matching paragraph or two... Best example I would tell people of the problem was users searching for things like: "Fall 2025 layoffs"... if the document existed, then things were being planned...

Ah Microsoft, security-last is still the thing, eh?

Re: Copilot broke audit logs, but Microsoft won't tell customers

#36

Post by catmanjan »

lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine...

Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in Google

Re: Copilot broke audit logs, but Microsoft won't tell customers

#37

Post by ath3nd »

verandaguy wrote: post #23
nzeid wrote: post #3

Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. What was their bug fix? Shadow prompts?

I'm very sceptical of using shadow prompts (or prompts of any kind) as an actual security/compliance control or enforcement mechanism. These things should be done using a deterministic system.

I bet you are a fan of OpenAI's groundbreaking study mode feature.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#38

Post by ecb_penguin »

db48x wrote: post #5
jayofdoom wrote: post #2

Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.

Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit l…

> CVEs are for vulnerabilities that are common across multiple products from multiple sources.

This is absolutely not true. I have no idea where you came up with this.

> Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit log entries.

That's not at all what the article says.

> That’s the worst design I could imagine!

Ok, well, that's not how they designed it.

> This is just engineering 101.

Where is the class for reading 101?

Re: Copilot broke audit logs, but Microsoft won't tell customers

#40

Post by gpm »

Spooky23 wrote: post #28
lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

No, it accesses data with the users privilege.

Are you telling me I, a normal unprivileged user, have a way to read files on windows that bypasses audit logs?
Post reply on HN