Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are a 400-year-long confidence trick

tomrenner.com

Post reply on HN
Print view View original

131–140 of 202 posts

Re: LLMs are a 400-year-long confidence trick

#131

Post by 7777332215 »

mpweiher wrote: post #117
krystofee wrote: post #32

I disagree with the "confidence trick" framing completely. My belief in this tech isn't based on marketing hype or someone telling me it's good – it's based on cold reality of what I'm shipping daily. The productivity gains I'm seeing right now are unprecedented. Even a year ago this wouldn't have been possible, it really feels like an inflection point. I'm seeing legitimate 10x gains because I'm not writing code any…

> I'm seeing legitimate 10x gains... Self-reports on this have been remarkably unreliable.

0.05x to 0.5x

Re: LLMs are a 400-year-long confidence trick

#132

Post by ACCount37 »

latexr wrote: post #125
ACCount37 wrote: post #110

Earlier quoted context omitted.

And there could be a teapot in an orbit around the Sun. Do we have any evidence for that being the case though? So far, there's a distinct lack of "wall" to be seen - and a lot of the proposed "fundamental" limitations of LLMs were discovered to be bogus with interpretability techniques, or surpassed with better scaffolding and better training.

> And there could be a teapot in an orbit around the Sun. I think you’re confused. You are the one making the extraordinary claim, the burden of proof is on you. You asserted LLMs have a finite number of steps to go to reach (overcome?) human limits. You don’t know that. It hasn’t happened. You can’t prove it. I, on the other hand, merely pointed out that is not a certainty. Your teapot argument works against you.

The case against "hard wall": every "hard wall" that was predicted so far was bypassed, and the measured performance keeps going up steadily.

The case for "hard wall": wishful thinking.

Re: LLMs are a 400-year-long confidence trick

#133

Post by runarberg »

ozim wrote: post #91
satisfice wrote: post #60

Earlier quoted context omitted.

I am hitting the keys, and I call bullshit. Yes, the technology is interesting and useful. No, it is not a “10x” miracle.

I call "AGI" or "100x miracle" a bullshit but still existing stuff definitely is "10x miracle".

This is a known sales trick, called door-in-the face. First you introduce your victim to an outrageous claim, and then follow with a more modest and more reasonable sounding claim.

In truth neither claims are reasonable, but because of the door in the face, the victim is more susceptible the the latter claim. Without the more outrageous claim it is unlikely the victim would have believed the latter claim.

In reality, both "AGI" and "100x miracle" AND the "10x miracle" are all outrageous claims, and I call bullshit on all of them.

Re: LLMs are a 400-year-long confidence trick

#134

Post by bodge5000 »

ACCount37 wrote: post #88
bodge5000 wrote: post #69

Earlier quoted context omitted.

> The LLM's are clearly useful for many things I don't think that's of any doubt. Even beyond programming, imo especially beyond programming, there are a great many things they're useful for. The question is; is that worth the enormous cost of running them? NFT's were cheap enough to produce and that didn't really scale depending on the "quality" of the NFT. With an LLM, if you want to produce something at the same s…

Yeah. Obviously. Duh. That's why we keep doing it. Opus 4.5 saved me about 10 hours of debugging stupid issues in an old build system recently - by slicing through the files like a grep ninja and eventually narrowing down onto a thing I surely would have missed myself. If I were to pay for the tokens I used at API pricing, I'd pay about $3 for that feat. Now, come up with your best estimate: what's the hourly wage of…

So could the company hiring you to do that work fire you and just use Opus instead? If no, then you cannot compare an engineers salary to what Opus costs, because the engineer is needed anyway.

> And API costs at all major providers represent selling the model with a good profit margin.

Though we don't know for certain, this is likely false. At best, it's looking like break even, but if you look at Anthropic, they cap their API spend at just $5,000 a month, which sounds like a stop loss. If it were making a good profit, they'd have no reason to have a stop loss (and certainly not that low).

> Yeah. Obviously. Duh. That's why we keep doing it.

I don't think so. I think what is promised is what keeps spend on it so high. I'd imagine if all the major AI companies were to come out and say "this is it, we've gone as far as we can", investment would likely dry up

Re: LLMs are a 400-year-long confidence trick

#135

Post by someuser484848 »

pancsta wrote: post #128
someuser484848 wrote: post #127

What? The comparison to the confidence trick from 400 years ago already stops at the second point? Why call this article that way if you are not going to bring up any parallels beyond... the extremely weak link of "building trust". You have not actually made clear how mechanical calculators were a scam. Ironically, this article feels like it was written by an LLM. Just a baseless opinion.

You missed the point. > Simply put, these companies have fallen for a confidence trick. They have built on centuries of received wisdom about the efficacy and reliability of computers, and have been drawn in by highly effective salespeople selling scarcely-believable technological wonders. Calculators are ok, but LLMs are not calculators.

I see what you mean - it wasn't intended to be a parallel to mechanical calculators.

However, the title implies that they were a trick - otherwise why is the "confidence trick" 400 years old?

I feel like this kind of imprecise use of language is what makes it difficult to interact with LLMs in a meaningful way - perhaps that is the reason the author seems to dismiss the value of them.

Re: LLMs are a 400-year-long confidence trick

#136

Post by ozlikethewizard »

latexr wrote: post #70
remus wrote: post #55

Earlier quoted context omitted.

The point of the article is to paint LLMs as a confidence trick, the keyword being trick. If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick? If a street performer was doing the cup and ball scam, but I actually won and left with more money than I started with then I'd say that's a pretty bad trick! Of course it is a little more nuanced than this and I would agree that s…

> If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick? Yes, yes you can. As I’ve mentioned elsewhere on this thread: > When a con man sells you a cheap watch for an high price, what you get is still useful—a watch that tells the time—but you were also still conned, because what you paid for is not what was advertised. You overpaid because you were tricked about what you w…

And at a cost im not sure most fully understand. We've allowed these companies to externalise all the negative outcomes. Now were seeing consumer electronics stock dry up, huge swaths of raw resources used, massive invasions of privacy, all so this one guy can do his corpo job 10x faster? Nah im good.

Re: LLMs are a 400-year-long confidence trick

#137

Post by pousada »

latexr wrote: post #106
pousada wrote: post #74

Earlier quoted context omitted.

It would make every marketing department and basically every startup founder conmen too. While I don’t completely disagree with that framing it’s not really helpful.

No, that is not true. Coca-Cola doesn’t advertise itself as a cure for cancer. Dropbox doesn’t advertise itself as a tax-filing application. Theranos on the other hand… That was a con and the founder was prosecuted. And again, Sam Altman has a history of deceit. https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-...

Coca-Cola advertises itself as a “happiness machine” as something that brings people together, builds communities, creates love and friendship.

The latest slogan i can remember is “open happiness”.

I like fizzy sugar drinks as much as the next guy but Coca-Cola is not liquid happiness.

Edit: Dropbox marketing slogan is “for all things worth saving”

Re: LLMs are a 400-year-long confidence trick

#138

Post by bodge5000 »

ACCount37 wrote: post #99
falloutx wrote: post #77

Earlier quoted context omitted.

All of the professions its trying to replace are very much bottom end of the tree, like programmers, designers, artists, support, lawyers etc. While you can easily already replace management and execs with it already and save 50% of the costs, but no one is talking about that. At this point the "trick" is to scare white collar knowledge workers into submission with low pay and high workload with the assumption that A…

"Just replace management and execs with AI" is an elaborate wagie cope. "Management and execs" are quite resistant to today's AI automation - and mostly for technical reasons. The main reason being: even SOTA AIs of today are subhuman at highly agentic tasks and long-horizon tasks - which are exactly the kind of tasks the management has to handle. See: "AI plays Pokemon", AccountingBench, Vending-Bench and its "real…

> even SOTA AIs of today are subhuman at highly agentic tasks and long-horizon tasks

This sounds like a lot of the work engineers do as well, we're not perfect at it (though execs aren't either), but the work you produce is expected to survive long term, thats why we spend time accounting for edge cases and so on.

Case in point; the popularity of docker/containerization. "It works on my machine" is generally fine in the short term, you can replicate the conditions of the local machine relatively easily, but doing that again and again becomes a problem, so we prepare for that (a long-horizon task) by using containers.

Re: LLMs are a 400-year-long confidence trick

#139

Post by latexr »

pousada wrote: post #137
latexr wrote: post #106

Earlier quoted context omitted.

No, that is not true. Coca-Cola doesn’t advertise itself as a cure for cancer. Dropbox doesn’t advertise itself as a tax-filing application. Theranos on the other hand… That was a con and the founder was prosecuted. And again, Sam Altman has a history of deceit. https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-...

Coca-Cola advertises itself as a “happiness machine” as something that brings people together, builds communities, creates love and friendship. The latest slogan i can remember is “open happiness”. I like fizzy sugar drinks as much as the next guy but Coca-Cola is not liquid happiness. Edit: Dropbox marketing slogan is “for all things worth saving”

Slogans are not promises, they are vague feelings. In the case of Coca-Cola, I know someone who might literally agree with the happiness part of it (though I certainly wouldn’t).

The promises of Theranos and LLMs are concrete measurable things we can evaluate and report where they succeed, fall short, or are lies.

Re: LLMs are a 400-year-long confidence trick

#140

Post by schnitzelstoat »

bodge5000 wrote: post #134
ACCount37 wrote: post #88

Earlier quoted context omitted.

Yeah. Obviously. Duh. That's why we keep doing it. Opus 4.5 saved me about 10 hours of debugging stupid issues in an old build system recently - by slicing through the files like a grep ninja and eventually narrowing down onto a thing I surely would have missed myself. If I were to pay for the tokens I used at API pricing, I'd pay about $3 for that feat. Now, come up with your best estimate: what's the hourly wage of…

So could the company hiring you to do that work fire you and just use Opus instead? If no, then you cannot compare an engineers salary to what Opus costs, because the engineer is needed anyway. > And API costs at all major providers represent selling the model with a good profit margin. Though we don't know for certain, this is likely false. At best, it's looking like break even, but if you look at Anthropic, they ca…

But now instead of spending 10 hours working on that, he can go and work on something else that would otherwise have required another engineer.

It's not going to mean they can employ 0 engineers, but maybe they can employ 4 instead of 5 - and a 20% reduction in workforce across the industry is still a massive change.

Post reply on HN