Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Copilot broke audit logs, but Microsoft won't tell customers

pistachioapp.com

Post reply on HN
Print view View original

101–110 of 324 posts

Re: Copilot broke audit logs, but Microsoft won't tell customers

#101

Post by TheRoque »

devjab wrote: post #93
TheRoque wrote: post #49

In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.

Unless you turn telemetry off (and believe they respect it) your entire filestructure, error and metadata will be shipped to Microsoft with no audit log available, simply by using VSCode. Which is frankly what copilot is doing here, except it's doing it on your 365 documents. I'm personally less concerned about Microsoft's impact on safety in terms of software development than I am with how all my data is handled by…

Sending some telemetry metadata on private servers is vastly different from sending code chunks and in some cases private environnement variables. On top of that, there are already many exploits and failures related to these tools which one again don't compare to simple telemetry. And I'm not even talking about the ethics of reproduced code without proper attribution, which is a different subject.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#102

Post by nyc_data_geek »

mlyle wrote: post #71
AdieuToLogic wrote: post #65

Earlier quoted context omitted.

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical…

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

Where do you think training data comes from

Re: Copilot broke audit logs, but Microsoft won't tell customers

#103

Post by trinsic2 »

lokar wrote: post #12
ceejayoz wrote: post #11

Earlier quoted context omitted.

> That can’t be right, can it? https://knowyourmeme.com/memes/james-franco-first-time

lol. I’ve avoided MS my entire (30+ year) career. Every now and then I’m reminded I made the right choice.

I woke up to MS in 2023[0]. Never again.

[0]: https://www.scottrlarson.com/publications/publication-transi...

Re: Copilot broke audit logs, but Microsoft won't tell customers

#104

Post by AdieuToLogic »

jpcosta wrote: post #83
MobiusHorizons wrote: post #80

Earlier quoted context omitted.

This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI?(process, legal, security, insurance etc) We have process and legal agreements in place to deal with humans that work for a business making mistakes. We need analogs for AI if we want to use it in similar ways.

What was the answer? Asking for a vp friend

>>> Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable."

>> This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI?

> What was the answer? Asking for a vp friend

This is a difficult issue to tackle, no doubt. What follows drifts into the philosophical realm by necessity.

Software exists to provide value to people. Malicious software qualifies as such due to the desires of the actors which produce same, but will no longer be considered here as this is not germane.

AI is an umbrella term for numerous algorithms having wide ranging problem domain applicability and often can approximate near-optimal solutions using significantly less resources than other approaches. But they are still algorithms, capable of only one thing - execute their defined logic.

Sometimes this logic can produce results similar to be what a person would in a similar situation. Sometimes the logic will produce wildly different results. Often there is significant value when the logic is used appropriately.

In all cases AI algorithms do not possess the concept of understanding. This includes derivatives of understanding such as:

  - empathy
  - integrity
  - morals
  - right
  - wrong
Which brings us back to part of the first quoted post:

  To quote IBM, "A computer can never be held accountable."
Accountability requires justification of actions taken or lack thereof, which demands the ability to explain why said actions were undertaken relative to other options, and implies a potential consequence be imposed by an authority.

Algorithms can partially "justify their output" via strategic logging, but that's about it.

Which is why "a computer can never be held accountable." Because it is a machine, executing the instructions ultimately initiated by one or more persons whom can be held accountable.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#107

Post by fulafel »

jayofdoom wrote: post #2

Generally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.

Not exactly.

There are several CVE numbering authorities and some of them (including the original MITRE, national CERTs etc), accept submissions from anyone, but there's evaluation and screening. Since Microsoft is their own CNA, most of them probably wouldn't issue a MS CVE without some kind of exceptional reason.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#108

Post by hitarpetar »

mlyle wrote: post #71
AdieuToLogic wrote: post #65

Earlier quoted context omitted.

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical…

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

I find this take to be purely misanthropic. we are more than stochastic parrots

Re: Copilot broke audit logs, but Microsoft won't tell customers

#109

Post by ygritte »

mlyle wrote: post #71
AdieuToLogic wrote: post #65

Earlier quoted context omitted.

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical…

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

I'm getting so tired of this dumb kind of non-argument. You can't defend LLMs on their own merits, so you try to make them look smarter by throwing shade on humans. That's a non sequitur and whataboutism.

Re: Copilot broke audit logs, but Microsoft won't tell customers

#110

Post by fulafel »

immibis wrote: post #54
db48x wrote: post #5

Earlier quoted context omitted.

Fun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit l…

More accurately, CVEs are for vulnerabilities that may be present on many systems. Then, the CVE number is a reference point that helps you when discussing the vulnerability, like asking whether it's present on a particular system, or what percentage of systems are patched. This vulnerability was only present on one system, so it doesn't need a CVE number. It could have a Microsoft-assigned bug number, but it doesn't…

This may be a stated reason but it's questionable logic. There are of course many cases where people need to reference and discuss this vulnerability and its impact.
Post reply on HN