Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

301–310 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#301

Post by bitfilped »

planb wrote: post #158

I am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a…

I know ethics aren't high up on the list of things we're taught about in tech, so I'd like to take a moment and point out that it's your moral responsibility to remove yourself from a project like this (or the company doing it.)

Re: Copilot broke audit logs, but Microsoft won't tell customers

#302

Post by AdieuToLogic »

Finbel wrote: post #296
anticodon wrote: post #128

Earlier quoted context omitted.

I'm pretty sure there're thousands of blog posts and books describing creation of a complete flashcards application in all popular programming languages and on all popular frameworks.

There're thousands of blog posts and books describing the vast majority of code most software developers write on a day to day basis. 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel.

> 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel.

That's like saying 99.99% of the food people eat consists of protein, carbohydrates, fats, and/or vegetables and therefore isn't novel. The implication being a McDonald's Big Mac and fries is the same as a spinach salad.

The only way someone could believe all food is the same as a Big Mac and fries is if this is all they ate and knew nothing else.

Hyperbole never ends well and neither does assuming novelty requires rarity or uniqueness, as distinct combinations of programmatic operations which deliver value in a problem domain is the very definition "new in an interesting way."

Just like how Thai noodles have proteins, carbohydrates, fats, and/or vegetables, yet are nothing like a Big Mac and fries.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#303

Post by fulafel »

thombles wrote: post #55
jayofdoom wrote: post #2

Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.

Is there value in requesting a CVE for a service that only Microsoft runs? What's a user supposed to do with that?

CVEs are supposed to be unambigous references to vulnerabilities for communication, nothing more. So you can say stuff like "this happened was before CVE-XXXX was fixed, do we need to notify anyone about the risk of undetected insider info access?"

Re: Copilot broke audit logs, but Microsoft won't tell customers

#304

Post by scott_w »

hannasanarion wrote: post #300
scott_w wrote: post #275

Earlier quoted context omitted.

> If so, then as the wise man says: "well, there‘s your problem!" Of course, I wouldn't defend this! To be clear, it's not possible to know how every AI Agent works, I just go off what I've seen when a company promises to unlock analytics insights on your data: usually by plugging directly into the Prod DB and having your data analysts complain whenever the engineers change the schema. > we should not judge the poten…

Having the agent plugged into the DB doesn't mean the agent can see everything in the DB. If that plug includes an automatic "where current user has access" filter, then the agent can't know anything the user can't know. That's what the bank agent analogy was meant to tell you. The agent has a direct line to the prod DB through their computer terminal, but every session they open is automatically constrained to the a…

> Having the agent plugged into the DB doesn't mean the agent can see everything in the DB. If that plug includes an automatic "where current user has access" filter, then the agent can't know anything the user can't know.

It depends on how it's plugged-in. If you just hand it a connection and query access then what exactly stops it? In a lot of SaaS systems, there's only the "application" user, which is restricted via queries within the API.

You can create a user in the DB per user of your application but this isn't free. Now you have the operational problem of managing your permissions, not via application logic, but subject to the rules and restrictions of your DBMS.

You can also create your own API layer on top, however this also comes with constraints of your API and adding protections on your query language.

None of this is impossible but, given what I've seen happen in the data analytics space, I can tell you that I know which option business leaders opt for.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#305

Post by flir »

tough wrote: post #267
flir wrote: post #238

Earlier quoted context omitted.

"the AI did it" is going to be the new "somebody hacked my facebook account" I wish I had a way of ensuring culpability remains with the human who published the text, regardless of who/what authored it.

if you're in a regulated field like law or medicine and you fuck up signing some AI slop with your name, you should loose your license at the very least tools are fine to use, personal responsability is still required. Companies already fuck up with this too much

I think it needs to be a cultural expectation. I don't know how we get there, though.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#307

Post by Finbel »

AdieuToLogic wrote: post #302
Finbel wrote: post #296

Earlier quoted context omitted.

There're thousands of blog posts and books describing the vast majority of code most software developers write on a day to day basis. 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel.

> 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel. That's like saying 99.99% of the food people eat consists of protein, carbohydrates, fats, and/or vegetables and therefore isn't novel. The implication being a McDonald's Big Mac and fries is the same as a spinach salad. The only way someone could believe all food is the same as a Big Mac and fries is if this is all…

Calling using a proven tool (LLM agents) to generate code that provide real business value a "fatal flaw" was already hyperbole.

The equivalent of not using LLMs in your workflow as a software engineer today isn't eating whole foods. That might have been true a year ago, but today it's becoming more and more equivalent to a fruit only diet.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#308

Post by thunky »

hannasanarion wrote: post #298
thunky wrote: post #243

Earlier quoted context omitted.

> You'd have to reindex the metadata (roles access), which may be substantial if you have a complex enough schema with enough users/roles. Right, but this compare this to the original proposal: > A basic implementation will return the top, let's say 1000, documents and then do the more expensive access check on each of them Using an index is much better than that. And it should be possible to update the index without…

I don't understand how "using an index" is a solution to this problem. If you're doing search, then you already have an index. If you use your index to get search results, then you will have a mix of roles that you then have to filter. If you want to filter first, then you need to make a whole new search index from scratch with the documents that came out of the filter. You can't use the same indexing information fro…

> I don't understand how "using an index" is a solution to this problem. If you're doing search, then you already have an index

It's a separate index.

You store document access rules in the metadata. These metadata fields can be indexed and then use as a pre-filter before the vector search.

> I don't think Elastic supports this

https://www.elastic.co/docs/solutions/search/vector/knn#knn-...

Re: Copilot broke audit logs, but Microsoft won't tell customers

#309

Post by Karrot_Kream »

scott_w wrote: post #259
mh- wrote: post #253

Earlier quoted context omitted.

I don't know how to say this less flippantly, and I honestly tried: you could have simply posted a comment phrased as a question, and 20 people would have jumped in to answer. (To your point, >15 of them would have had different answers and the majority would have been materially wrong, but still.)

So let me be more direct. The part I'm not confident I'm correct in is this: > AI Agents, instead, directly access the database However, I don't think I'd be too far off the mark given many systems work like this (analytics tools typically hook into your DB, to the chagrin of many an SRE/DevOps) and it's usually marketed as the easy solution. Also, I've since read a few comments and it appears I'm pretty fucking clos…

The correct way to setup an analytics tool is to point it to an analytics db that is a replica of your main DB. It's a pretty common part of an HA setup to replicate your primary to an actual hot read replica and a cold analytics store. This way the analytics tool queries your analytics store and doesn't put load on your hot primary or hot read replica.

> I'd say most of my comment is substantively correct, with a disclaimer on an (important) point, where I'd be happy to be corrected.

I read this and feel that you still want imaginary internet points for something that is, at best, directionally correct. To me it seems your desire for internet points urged you to post a statement and not a question. I imagine most of HN is just statements that are overconfident bluster by only directionally correct statements which create the cacophony of this site.

Post reply on HN