Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

111–120 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#111

Post by ygritte »

AdieuToLogic wrote: post #78
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. Of note too is that the same "systems made out of meat" have been producing content satisfying the strict mathematical model for decades and continue to do so beyond the capabilities of the aforementioned algorithms.

+1. Also, those systems made out of meat were the ones that discovered the strict mathematical models in the first place and the rules by which they work.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#112

Post by ozim »

cookiengineer wrote: post #85
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

The difference: Meatbags created something like an education system, where the track record in it functions as a ledger for hiring companies. There is no such thing for AI. No ledger, no track record, no reproducibility.

Batter. Meatbags created liability system where if meatbag harms others they go to jail.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#113

Post by mlyle »

ygritte wrote: post #109
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

I'm getting so tired of this dumb kind of non-argument. You can't defend LLMs on their own merits, so you try to make them look smarter by throwing shade on humans. That's a non sequitur and whataboutism.

Nah-- I feel like I have my eyes pretty wide open about the shortcomings of LLMs (but still find them useful often).

But any argument seeking to dunk on LLMs needs to not also apply equally to the alternative (humans).

Re: Copilot broke audit logs, but Microsoft won't tell customers

#114

Post by Gud »

cookiengineer wrote: post #85
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

The difference: Meatbags created something like an education system, where the track record in it functions as a ledger for hiring companies. There is no such thing for AI. No ledger, no track record, no reproducibility.

On the other hand, some of the most capable meat bags said fuck you to your record keeping system and dropped out.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#115

Post by mlyle »

AdieuToLogic wrote: post #78
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. Of note too is that the same "systems made out of meat" have been producing content satisfying the strict mathematical model for decades and continue to do so beyond the capabilities of the aforementioned algorithms.

It's usually not the same pile of meat defining the problem and solving the problem.

Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains.

The style of argument you made about the nature of the machinery used applies just as well (maybe better) to humans. To get a valid argument, we'll need to be more nuanced.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#116

Post by galaxyLogic »

MobiusHorizons wrote: post #80
jdiff wrote: post #76

Earlier quoted context omitted.

Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable."

This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI?(process, legal, security, insurance etc) We have process and legal agreements in place to deal with humans that work for a business making mistakes. We need analogs for AI if we want to use it in similar ways.

My question is if I get some code from AI, save it to a file, then modify it or add some functions to it, can I still claim the copyright for it at the top of the file? Do I need to give the AI any credit?

I'm asking because I read somewhere that "AI produced output cannot be copyrighted". But what if I modify that output myself? I am then a co-creator, right, and I think I should have a right to some copyright protection.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#117

Post by ozim »

AdieuToLogic wrote: post #65
TheRoque wrote: post #49

In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical…

Sounds like we are in the end game for „move fast and break things”. Doesn’t feel like we can invent something that moves even faster and breaks more.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#118

Post by Hilift »

lokar wrote: post #8

Wait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?

In Windows, if a process has Backup privilege it can bypass any permissions, and it is not audited by default due to it would create too much audit volume by actual backup applications. Any process that has this privilege can use it, but the privilege is disabled by default, so it would require deliberate enablement. It is fairly easy to enable in managed code like C#. Same goes for Restore privilege.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#119

Post by zimpenfish »

jdiff wrote: post #76
mlyle wrote: post #71

Earlier quoted context omitted.

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable."

> "A computer can never be held accountable."

cf the Post Office scandal in the UK which was partly helped along by the 1999 change in law[1] which repealed the 1984 stance that "computer evidence is not permissible unless it is shown to be working correctly at the time"[0]; i.e. that a computer was now presumed to be working correctly and it was up to the defence to prove otherwise.

[0] https://www.legislation.gov.uk/ukpga/1984/60/section/69/1991...

[1] https://www.legislation.gov.uk/ukpga/1999/23/section/60/1999...

Re: Copilot broke audit logs, but Microsoft won't tell customers

#120

Post by Finbel »

AdieuToLogic wrote: post #65
TheRoque wrote: post #49

In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical…

Have you not used commercial LLMs to generate program source code? You describe it as if it's an almost unsolvable problem which might have been reasonable 2 years ago but I just used gpt-5 to generate a complete NextJS application for flashcards.

I've literally been employing nondeterministic content generation based on statistical relevance defined by an unknown training data, to repeatably produce content satisfying a strict mathematical model for months now.

Post reply on HN