Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

161–170 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#161

Post by carschno »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> I am having a really hard time communicating this problem to executives

When you hit such a wall, you might not be failing to communicate, nor them failing to understand. In reality, said executives have probably chosen to ignore the issue, but also don't want to take accountability for the eventual leaks. So "not understanding" is the easiest way to blame the engineers later.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#162

Post by philipallstar »

VitalKoshalew wrote: post #125
ocdtrekkie wrote: post #43

Earlier quoted context omitted.

I would say "insecure by default". I talked to some Microsoft folks around the Windows Server 2025 launch, where they claimed they would be breaking more compatibility in the name of their Secure Future Initiative. But Server 2025 will load malicious ads on the Edge start screen[1] if you need to access a web interface of an internal thing from your domain controller, and they gleefully announced including winget, a…

Servers don't have Desktop GUI, so there is no way you can run a browser on a real server installation. That's done specifically to limit the attack surface. This applies to almost all Windows Server roles except very few such as ADFS which Microsoft is struggling to migrate for decades. Definitely to the root of all security - AD DC. If you've elected to create a Frankenstein of a domain controller and a desktop/gam…

Windows technicians are only proficient in ClickOps, so, yes. It has a GUI.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#163

Post by smolder »

deadbabe wrote: post #146
eleveriven wrote: post #136

This is exactly the kind of issue that makes trust in large vendors like Microsoft feel more like a gamble than a guarantee

So who do you trust? A small mom and pop software biz?

Generally speaking, yes, relative to the giants. Smallsoft co at least isn't going to monetize every bit of knowledge about you directly. They'll probably leak stuff to chatgpt and so on, but it depends on the business purpose whether I'd be worried about that.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#164

Post by gloosx »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

What do you mean a problem? It's an AI man. Just ask it what to do man. It's thinking, it's really really big thinking. Big thing which does all big thinking. The multi-modal reasoning deep big thinking bro. Security, permissions, thats so important for you?? We have AI. It does thinking. What else do you need?? Because two brains are better than one. Your backlog doesn’t stand a chance. Get speed when you need it. Depth when you don’t. Make one change. Copilot handles the rest. It's your code’s guardian angel. It's AI bro. AGI is coming tomorrow. Delegate like a boss. Access rights and all the complex things can wait.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#165

Post by krisoft »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale.

I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the potentially relevant list of documents you check which ones can the current user access. You only pass the ones over to the LLM which the user can see.

This is very similar to how banks provide phone based services. The operator on the other side of the line can only see your account details once you have authenticated yourself. They can't accidentally tell you someone else's account balance, because they themselves don't have access to it unless they typed in all the information you provide them to authenticate yourself. You can't trick the operator to provide you with someone else's account balance because they can't see the account balance of anyone without authenticating first.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#166

Post by lupusreal »

carschno wrote: post #161
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> I am having a really hard time communicating this problem to executives When you hit such a wall, you might not be failing to communicate, nor them failing to understand. In reality, said executives have probably chosen to ignore the issue, but also don't want to take accountability for the eventual leaks. So "not understanding" is the easiest way to blame the engineers later.

It doesn't even need to be blaming the engineers in this case, they can blame "the AI" and most people will accept that and let whatever incident happened slide. If somebody questions the wisdom of putting AI in such a position, they can be dismissed as not appreciating new technology (even though their concern is valid.)

Re: Copilot broke audit logs, but Microsoft won't tell customers

#167

Post by inejge »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> I am having a really hard time communicating this problem to executives

Cc Legal/Compliance could do wonders to their capacity to understand the problem. Caveat, of course, that the execs might be pissed off that some peon is placing roadblocks in the way of their buzzword-happy plan.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#168

Post by chaosite »

krisoft wrote: post #165
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

> This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. I don't understand why you think tracking user access rights would be infeasible and would not scale. There is a query. You search for matching documents in your vector database / index. Once you have found the pot…

> You search for matching documents in your vector database / index. Once you have found the potentially relevant list of documents you check which ones can the current user access. You only pass the ones over to the LLM which the user can see.

Sometimes the potentially relevant list of documents itself is a leak all by itself.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#169

Post by [deleted] »

lupusreal wrote: post #166
carschno wrote: post #161

Earlier quoted context omitted.

> I am having a really hard time communicating this problem to executives When you hit such a wall, you might not be failing to communicate, nor them failing to understand. In reality, said executives have probably chosen to ignore the issue, but also don't want to take accountability for the eventual leaks. So "not understanding" is the easiest way to blame the engineers later.

It doesn't even need to be blaming the engineers in this case, they can blame "the AI" and most people will accept that and let whatever incident happened slide. If somebody questions the wisdom of putting AI in such a position, they can be dismissed as not appreciating new technology (even though their concern is valid.)

[deleted]

Re: Copilot broke audit logs, but Microsoft won't tell customers

#170

Post by planb »

cryptonym wrote: post #160
planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

True, per user doesn't scale. Knowledge should be properly grouped and have rights on database, documents, and chatbot managed by groups. For instance specific user can use the Engineering chatbot but not the Finance one. If you fail to define these groups, feels like you don't have a solid strategy. In the end, if that's what they want, let them experience open knowledge.

Yeah. If you have knowledge stored in a structured form like that, you don't need an AI...
Post reply on HN