Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

281–290 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#281

Post by zo1 »

lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

Judging by what I've been seeing in the field in the last half-decade, this doesn't surprise me one bit. Zero forward thinking and comprehensive analysis of features before they are built, with tickets just being churned out by incessant meetings that only end because people get tired. And the devs just finish the tickets without ever asking why a feature is being built or how it actually has to interact with the rest of the system.

Multiply that by years, by changing project managers and endless UX re-writes, huge push for DEI over merit, junior & outsourced-heavy hires and forced promotions, and you end up getting this mess that is "technically" working and correct but no one can quantify the potential loss and lack of real progress that could have been made if actual competent individuals were put in charge.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#282

Post by ThrowMeAway1618 »

ecb_penguin wrote: post #38
db48x wrote: post #5

Earlier quoted context omitted.

Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit l…

> CVEs are for vulnerabilities that are common across multiple products from multiple sources. This is absolutely not true. I have no idea where you came up with this. > Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit log entries. That's not at all what the article says. > That’s the worst design I could imagine! Ok, well, that's not how they des…

>> CVEs are for vulnerabilities that are common across multiple products from multiple sources.

>This is absolutely not true. I have no idea where you came up with this.

Perhaps they asked Copilot?

Re: Copilot broke audit logs, but Microsoft won't tell customers

#283

Post by JTbane »

ThrowawayTestr wrote: post #56
TheRoque wrote: post #49

In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.

Companies won't use open source software because of licencing concerns but if you launder it through an LLM it's hunky-dory.

This is kind of not true. Companies will gladly use MIT-style license open source software for on-premises proprietary products, and everything but AGPL-style software for cloud products.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#284

Post by Spooky23 »

waffleiron wrote: post #69
Spooky23 wrote: post #33

Earlier quoted context omitted.

It’s a bug. He reported it, they fixed it. It is not a five alarm fire for HIPAA. HIPAA doesn’t require that all file access be logged at all. HIPAA also doesn’t require that a CVE be created for each defect in a product. End of the day, it’s a hand-wavy, “look at me” security blog. Don’t get too crazy.

I am more on the privacy side of things like HIPAA, but I would like to link the following. https://www.hhs.gov/sites/default/files/january-2017-cyber-n...

There’s discretion in reasonable and appropriate.

Biggest thing is to have plan and policy. I’d agree in general that more audit is better.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#285

Post by planb »

krisoft wrote: post #165
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the pot…

I’ll answer to this as a placeholder for all the „just do xyz“ replies:

Searching the whole index and then filtering is possible, but infeasible for large indexes where a specific user only has access to a few docs. And for diverse data sources (as we want to access), this would be really slow, many systems would need to be checked.

So, access rights should be part of the index. In that case, we are just storing a copy of the access rights, so this is prone to races. Besides that, we have multiple systems with different authorization systems, groups, roles, whatever. To homogenize this, we would need to store the info down to each individual user. Besides this, not all systems even support asking which users have access to resource Y, they only allow to ask „has X access to Y“.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#286

Post by planb »

giamma wrote: post #226
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

I believe most vector databases allow you to annotate vectors with additional metadata. Why not simply add as metadata the list of principals (roles/groups) who have access to the information (e.g. HR, executives) ? Then when a user makes a request to the chatbot, you expand the user identity to his/her principals (e.g. HR) and use those as implicit filtering criteria for finding the closest vectors in the database.…

I am in control of the vector database and the search index. I have no control over the different accessed data sources that don’t even allow to query access rights per resource (and just allow for can_access checks for a given user)

Re: Copilot broke audit logs, but Microsoft won't tell customers

#287

Post by tuple »

jdiff wrote: post #266
galaxyLogic wrote: post #116

Earlier quoted context omitted.

My question is if I get some code from AI, save it to a file, then modify it or add some functions to it, can I still claim the copyright for it at the top of the file? Do I need to give the AI any credit? I'm asking because I read somewhere that "AI produced output cannot be copyrighted". But what if I modify that output myself? I am then a co-creator, right, and I think I should have a right to some copyright prote…

First, a few disclaimers, I am not a lawyer and this is an actively evolving area. The answer that most aligns with current precedent to my knowledge is that the parts you modify are protected by your copyright, but the rest remains uncopyrightable. With the exception of any chunks generated that align with someone's existing copyrighted code, as long as those chunks are substantial and unique enough.

This is how I understand it as well. There was a case of a monkey taking a selfie and the photog attempting to assert copyright. Courts determined (if I remember correctly :p ) that only human works are copyrightable and therefore the monkey's selfie could not be copyrighted at all.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#288

Post by HelloImSteven »

immibis wrote: post #54
db48x wrote: post #5

Earlier quoted context omitted.

Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit l…

More accurately, CVEs are for vulnerabilities that may be present on many systems. Then, the CVE number is a reference point that helps you when discussing the vulnerability, like asking whether it's present on a particular system, or what percentage of systems are patched. This vulnerability was only present on one system, so it doesn't need a CVE number. It could have a Microsoft-assigned bug number, but it doesn't…

But this isn't a problem on one system, it's potentially a problem in any system with Copilot enabled. It's akin to a vulnerability in a software library (which often means a separate CVE for every affected product, not just one for the library). CVEs also limited to issues impacting multiple systems; even if a vulnerability only affects one product, ideally a CVE should get made. The 'common' aspect is the shared reporting standard. See my other comment on this thread for more on that, or Redhat's explanation here: https://www.redhat.com/en/topics/security/what-is-cve

Re: Copilot broke audit logs, but Microsoft won't tell customers

#289

Post by fc417fc802 »

Hilift wrote: post #118
lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

In Windows, if a process has Backup privilege it can bypass any permissions, and it is not audited by default due to it would create too much audit volume by actual backup applications. Any process that has this privilege can use it, but the privilege is disabled by default, so it would require deliberate enablement. It is fairly easy to enable in managed code like C#. Same goes for Restore privilege.

... and? In an audited environment you'd carefully vet how the backups work. That functionality is inside the security boundary so to speak.

I don't believe it's integrated with (any bypass of) auditing but the same "ignore permissions" capability exists on Linux as CAP_DAC_READ_SEARCH and is primarily useful for the same sort of tasks.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#290

Post by polishdude20 »

Drakim wrote: post #194
falcor84 wrote: post #192

Earlier quoted context omitted.

I'm unclear on what the "flaw" is - isn't this precisely the "feature" that search engines provide to both sides and that site owners put a ton of SEO effort into optimizing?

If you have public documents, you can obviously let a public search engine index them and show previews. All is good. If you have private documents, you can't let a public search engine index and show previews of those private documents. Even if you add an authentication wall for normal users if they try to open the document directly. They could still see part of the document in google's preview. My explanation sound…

That's not necessarily what happened in the article. He wasn't able to access private docs. He was just able to tell Copilot to not send an audit log.
Post reply on HN