Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I’m worried that they put co-pilot in Excel

simonwillison.net

Post reply on HN
Print view View original

231–240 of 346 posts

Re: I’m worried that they put co-pilot in Excel

#231

Post by fancyfredbot »

tl wrote: post #64
fancyfredbot wrote: post #36

This is transparent nonsense. People are very very happy to introduce errors into excel spreadsheets without any help from AI. Financial statements are correct because of auditors who check the numbers. If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did woul…

> If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit). Volume matters. The single largest problem I run into: AI can generate slop faster than anyone can evaluate it.

If nobody can evaluate it then nobody will sign it off.

Re: I’m worried that they put co-pilot in Excel

#232

Post by sjsdaiuasgdia »

Esophagus4 wrote: post #192
bigstrat2003 wrote: post #182

Earlier quoted context omitted.

> Because it doesn’t have to be as accurate as a human to be a helpful tool. I disagree. If something can't be as accurate as a (good) human, then it's useless to me. I'll just ask the human instead, because I know that the human is going to be worth listening to.

Autopilot in airplanes is a good example to disprove that. Good in most conditions. Not as good as a human. Which is why we still have skilled pilots flying planes, assisted by autopilot. We don’t say “it’s not as good as a human, so stuff it.” We say, “it’s great in most conditions. And humans are trained how to leverage it effectively and trained to fly when it cannot be used.”

The autopilots in aircraft have predictable behaviors based on the data and inputs available to them.

This can still be problematic! If sensors are feeding the autopilot bad data, the autopilot may do the wrong thing for a situation. Likewise, if the pilot(s) do not understand the autopilot's behaviors, they may misuse the autopilot, or take actions that interfere with the autopilot's operation.

Generative AI has unpredictable results. You cannot make confident statements like "if inputs X, Y, and Z are at these values, the system will always produce this set of outputs".

In the very short timeline of reacting to a critical mid-flight situation, confidence in the behavior of the systems is critical. A lot of plane crashes have "the pilot didn't understand what the automation was doing" as a significant contributing factor. We get enough of that from lack of training, differences between aircraft manufacturers, and plain old human fallibility. We don't need to introduce a randomized source of opportunities for the pilots to not understand what the automation is doing.

Re: I’m worried that they put co-pilot in Excel

#233

Post by sjsdaiuasgdia »

stavros wrote: post #226
sjsdaiuasgdia wrote: post #219

Earlier quoted context omitted.

RE: the calculator screenshot - it's still reliable because the same answer will be produced for the same inputs every time. And the behavior, though possibly confusing to the end user at times, is based on choices made in the design of the system (floating point vs integer representations, rounding/truncating behavior, etc). It's reliable deterministic logic all the way down.

> I've never seen a calculator come up with the wrong answer when adding two numbers. 1.00000001 + 1 doesn't equal 2, therefore the claim is false.

Sure it does, if you have made a system design decision about the precision of the outputs.

At the precision the system is designed to operate at, the answer is 2.

Re: I’m worried that they put co-pilot in Excel

#234

Post by simonw »

NumberCruncher wrote: post #224

Is this not the guy who is on the payroll of Anthropic? Not because he is wrong, but because there is so much marketing going on in this space nowadays.

Who, me? I'm still independent - I have a disclosures section on my blog here: https://simonwillison.net/about/#disclosures

Anthropic sometimes give me free credits (so I can try out preview features) and gave me a ticket to their conference a few months ago.

Re: I’m worried that they put co-pilot in Excel

#235

Post by [deleted] »

stavros wrote: post #225
faeyanpiraat wrote: post #222

Earlier quoted context omitted.

You mixed up correctness and reliability. The ios calculator will make the same incorrect calculation, but reliably, every time.

Don't move the goalposts. The claim was: > I've never seen a calculator come up with the wrong answer when adding two numbers. 1.00000001 + 1 doesn't equal 2, therefore the claim is false.

[deleted]

Re: I’m worried that they put co-pilot in Excel

#237

Post by wat10000 »

stavros wrote: post #185
wat10000 wrote: post #169

Earlier quoted context omitted.

Programs can be very close to 100% reliable when made well. In my life, I've never seen `sort` produce output that wasn't properly sorted. I've never seen a calculator come up with the wrong answer when adding two numbers. I have seen filesystems fail to produce the exact same data that was previously written, but this is something that happens once in a blue moon, and the process is done probably millions of times a…

> Programs can be very close to 100% reliable when made well. This is a tautology. > I've never seen a calculator come up with the wrong answer when adding two numbers. https://imgz.org/i6XLg7Fz.png > And technically, most bugs are predictable in theory, they just aren't known ahead of time. When we're talking about reliability, it doesn't matter whether a thing can be reliable in theory, it matters whether it's reli…

> > Programs can be very close to 100% reliable when made well. > This is a tautology.

No it's not. There are plenty of things that can't be 100% reliable no matter how well they're made. A perfect bridge is still going to break down and eventually fall apart. The best possible motion-activated light is going to have false positives and false negatives because the real world is messy. Light bulbs will burn out no matter how much care and effort goes into them.

In any case, unless you assert that programs are never made well, then your own statement disproves your previous statement that the reliability of programs is "far from 100%."

Plenty of software is extremely reliable in practice. It's just easy to forget about it because good, reliable software tends to be invisible.

Re: I’m worried that they put co-pilot in Excel

#238

Post by ryandrake »

yfontana wrote: post #227
ryandrake wrote: post #173

Earlier quoted context omitted.

I've been trying to open my mind and "give AI a chance" lately. I spent all day yesterday struggling with Claude Code's utter incompetence. It behaves worse than any junior engineer I've ever worked with: - It says it's done when its code does not even work, sometimes when it does not even compile. - When asked to fix a bug, it confidently declares victory without actually having fixed the bug. - It gets into this mo…

> - It says it's done when its code does not even work, sometimes when it does not even compile. > - When asked to fix a bug, it confidently declares victory without actually having fixed the bug. You need to give it ways to validate its work. A junior dev will also give you code that doesn't compile or should have fixed a bug but doesn't if they don't actually compile the code and test that the bug is truly fixed.

Believe me, I've tried that, too. Even after giving detailed instructions on how to validate its work, it often fails to do it, or it follows those instructions and still gets it wrong.

Don't get me wrong: Claude seems to be very useful if it's on a well-trodden train track and never has to go off the tracks. But it struggles when its output is incorrect.

The worst behavior is this "try things over and over" behavior, which is also very common among junior developers and is one of the habits I try to break from real humans, too. I've gone so far as to put into the root CLAUDE.md system prompt:

--NEVER-- try fixes that you are not sure will work.

--ALWAYS-- prove that something is expected to work and is the correct fix, before implementing it, and then verify the expected output after applying the fix.

...which is a fundamental thing I'd ask of a real software engineer, too. Problem is, as an LLM, it's just spitting out probabilistic sentences: it is always 100% confident of its next few words. Which makes it a poor investigator.

Re: I’m worried that they put co-pilot in Excel

#239

Post by wat10000 »

stavros wrote: post #225
faeyanpiraat wrote: post #222

Earlier quoted context omitted.

You mixed up correctness and reliability. The ios calculator will make the same incorrect calculation, but reliably, every time.

Don't move the goalposts. The claim was: > I've never seen a calculator come up with the wrong answer when adding two numbers. 1.00000001 + 1 doesn't equal 2, therefore the claim is false.

Sorry, but this annoys me. The claim might be false if I had made it after seeing your screenshot. But you don't know what I've seen in my life up to that point. The claim that all calculators are infallible would be false, but that's not the claim I made.

When a personal experience is cited, a valid counterargument would be "your experience is not representative," not "you are incorrect about your own experience."

Re: I’m worried that they put co-pilot in Excel

#240

Post by mrgoldenbrown »

Aeolun wrote: post #42
quixoticaxolotl wrote: post #39

I find the contrast between two narratives around technology use so fascinating: 1. We advocate automation because people like Brenda are error-prone and machines are perfect. 2. We disavow AI because people like Brenda are perfect and the machine is error-prone. These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the pro…

> We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.

They want to trust it, because then they can stop paying Brenda, save a few dollars, and buy a 3rd yacht.
Post reply on HN