Copilot broke audit logs, but Microsoft won't tell customers
pistachioapp.com
Copilot broke audit logs, but Microsoft won't tell customers
1–10 of 324 posts
Re: Copilot broke audit logs, but Microsoft won't tell customers
#2Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.
Re: Copilot broke audit logs, but Microsoft won't tell customers
#3Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs.
What was their bug fix? Shadow prompts?
Re: Copilot broke audit logs, but Microsoft won't tell customers
#4Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. What was their bug fix? Shadow prompts?
I'd hope that if a tool the LLM uses reveals any part of the file to the LLM it counts as a read by every user who sees any part of the output that occurred after that revelation was added to the context.
Re: Copilot broke audit logs, but Microsoft won't tell customers
#5Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.
Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria.
Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit log entries. That’s the worst design I could imagine! When they use an API to access a file or a url then the API should create the audit log. This is just engineering 101.
Re: Copilot broke audit logs, but Microsoft won't tell customers
#6Hardly have I ever seen corporate incentives so aligned to overhype the capabilities of a technology while it being so raw and unpolished as this one.
The bubble bursting will be epic.
Re: Copilot broke audit logs, but Microsoft won't tell customers
#7Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.
Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit l…
Huh, there are CVEs for windows components all the time, random example: https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
Including for end user applications, not libraries, another random example: https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
Re: Copilot broke audit logs, but Microsoft won't tell customers
#8Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions)
That can’t be right, can it?
Re: Copilot broke audit logs, but Microsoft won't tell customers
#9I am so tired of Microsoft cramming Copilot into everything. Search at $dayjob is completely borked right now. It shows a page of results, but the immediately pops up some warning dialog you cannot dismiss that Copilot can’t access some file “” or something. Every VSCode update I feel like I have to turn off Copilot in some new way. And now apparently it’ll be added to Excel as well. Thankfully I don’t have to use anything from Microsoft after work hours.