Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are a 400-year-long confidence trick

tomrenner.com

Post reply on HN
Print view View original

121–130 of 202 posts

Re: LLMs are a 400-year-long confidence trick

#121

Post by ACCount37 »

myrmidon wrote: post #97
ACCount37 wrote: post #75

Earlier quoted context omitted.

If that's the case, then, what's the wall? The "walls" that stopped AI decades ago stand no more. NLP and CSR were thought to be the "final bosses" of AI by many - until they fell to LLMs. There's no replacement. The closest thing to a "hard wall" LLMs have is probably online learning? And even that isn't really a hard wall. Because LLMs are good at in-context learning, which does many of the same things, and can do…

Agree completely with your position. I do think though that lack of online learning is a bigger drawback than a lot of people believe, because it can often be hidden/obfuscated by training for the benchmarks, basically. This becomes very visible when you compare performance on more specialized tasks that LLMs were not trained for specifically, e.g. playing games like Pokemon or Factorio: General purpose LLMs are lagg…

By now, I subscribe to "you're just training them wrong".

Pre-training a base model on text datasets teaches that model a lot, but it doesn't teach it to be good at agentic tasks and long horizon tasks.

Which is why there's a capability gap there - the gap companies have to overcome "in post" with things like RLVR.

Re: LLMs are a 400-year-long confidence trick

#122

Post by 112233 »

ACCount37 wrote: post #38
schnitzelstoat wrote: post #11

I agree that all the AI doomerism is silly (by which I mean those that are concerned about some Terminator-style machine uprising, the economic issues are quite real). But it's clear the LLM's have some real value, even if we always need a human-in-the-loop to prevent hallucinations it can still massively reduce the amount of human labour required for many tasks. NFT's felt like a con, and in retrospect were a con. T…

LLMs of today advance in incremental improvements. There is a finite amount of incremental improvements left between the performance of today's LLMs and the limits of human performance. This alone should give you second thoughts on "AI doomerism".

pole-vaulting records improve incrementally too. and there is finite distance left to the moon. without deep understanding and experience and numbers to back up the opinion, any progress seems about to reach arbitrary goals.

Re: LLMs are a 400-year-long confidence trick

#123

Post by ACCount37 »

falloutx wrote: post #115
ACCount37 wrote: post #99

Earlier quoted context omitted.

"Just replace management and execs with AI" is an elaborate wagie cope. "Management and execs" are quite resistant to today's AI automation - and mostly for technical reasons. The main reason being: even SOTA AIs of today are subhuman at highly agentic tasks and long-horizon tasks - which are exactly the kind of tasks the management has to handle. See: "AI plays Pokemon", AccountingBench, Vending-Bench and its "real…

Some management would be cut off when the time comes, Execs on the other hand are not there for work and are in due to personal relationships, so impossible to fire. If you think someone like lets say Satya Nadella can't be replaced by a bot which takes different input streams and then makes decisions, then you are joking. Even his recent end of 2025 letter was mostly written by AI.

If an AI exec reliably outperformed meatbag execs while demanding less $$$, many boards would consider that an upgrade. Why gamble on getting a rare high performance super-CEO when you can get a reliable "good enough"?

The problem is: we don't have an AI exec that would outperform a meatbag exec on average, let alone reliably. Yet.

Re: LLMs are a 400-year-long confidence trick

#124

Post by qsera »

obsoleetorr wrote: post #104
qsera wrote: post #100

Earlier quoted context omitted.

There is Intelligence and there is Imitation of Intelligence. LLMs do the latter. Talk to any model about deep subjects. You ll understand what I am saying. After a while it will start going around in circles. FFS ask it to make an original joke, and be amused..

> After a while it will start going around in circles. so like your average human > FFS ask it to make an original joke, and be amused.. let's try this one on you - say an original joke oh, right, you dont respond to strangers prompts, thus you have agency, unlike an LLM

>so like your average human

If an average human has seen and read all that is written till now, I bet that they can hold the conversation going for quite a long time...

>say an original joke

I asked an LLM if it had a good night with sweet dreams, It said, "I don't sleep and I only dream when I work!"

Re: LLMs are a 400-year-long confidence trick

#125

Post by latexr »

ACCount37 wrote: post #110
latexr wrote: post #93

Earlier quoted context omitted.

> If that's the case, then, what's the wall? I didn’t say that is the case, I said it could be. Do you understand the difference? And if it is the case, it doesn’t immediately follow that we would know right now what exactly the wall would be. Often you have to hit it first. There are quite a few possible candidates.

And there could be a teapot in an orbit around the Sun. Do we have any evidence for that being the case though? So far, there's a distinct lack of "wall" to be seen - and a lot of the proposed "fundamental" limitations of LLMs were discovered to be bogus with interpretability techniques, or surpassed with better scaffolding and better training.

> And there could be a teapot in an orbit around the Sun.

I think you’re confused. You are the one making the extraordinary claim, the burden of proof is on you.

You asserted LLMs have a finite number of steps to go to reach (overcome?) human limits. You don’t know that. It hasn’t happened. You can’t prove it.

I, on the other hand, merely pointed out that is not a certainty.

Your teapot argument works against you.

Re: LLMs are a 400-year-long confidence trick

#126

Post by [deleted] »

JacoboJacobi wrote: post #116
latexr wrote: post #98

Earlier quoted context omitted.

> The claims being made that are cited are not really in that camp though.. They literally are. Sam Altman has literally said multiple times this tech will cure cancer.

Such claims are not cited in the article. It may be possible to write a good article on the topic but this article could just as well be on the organic uptake of the PC and how most wealthy nontechnical people adopted a need for a PC on "cons" that preceded their ability to gain more worth than trouble.

[deleted]

Re: LLMs are a 400-year-long confidence trick

#127

Post by someuser484848 »

What? The comparison to the confidence trick from 400 years ago already stops at the second point? Why call this article that way if you are not going to bring up any parallels beyond... the extremely weak link of "building trust".

You have not actually made clear how mechanical calculators were a scam.

Ironically, this article feels like it was written by an LLM. Just a baseless opinion.

Re: LLMs are a 400-year-long confidence trick

#128

Post by pancsta »

someuser484848 wrote: post #127

What? The comparison to the confidence trick from 400 years ago already stops at the second point? Why call this article that way if you are not going to bring up any parallels beyond... the extremely weak link of "building trust". You have not actually made clear how mechanical calculators were a scam. Ironically, this article feels like it was written by an LLM. Just a baseless opinion.

You missed the point.

> Simply put, these companies have fallen for a confidence trick. They have built on centuries of received wisdom about the efficacy and reliability of computers, and have been drawn in by highly effective salespeople selling scarcely-believable technological wonders.

Calculators are ok, but LLMs are not calculators.

Re: LLMs are a 400-year-long confidence trick

#129

Post by cindyllm »

ACCount37 wrote: post #123
falloutx wrote: post #115

Earlier quoted context omitted.

Some management would be cut off when the time comes, Execs on the other hand are not there for work and are in due to personal relationships, so impossible to fire. If you think someone like lets say Satya Nadella can't be replaced by a bot which takes different input streams and then makes decisions, then you are joking. Even his recent end of 2025 letter was mostly written by AI.

If an AI exec reliably outperformed meatbag execs while demanding less $$$, many boards would consider that an upgrade. Why gamble on getting a rare high performance super-CEO when you can get a reliable "good enough"? The problem is: we don't have an AI exec that would outperform a meatbag exec on average, let alone reliably. Yet.

[dead]

Re: LLMs are a 400-year-long confidence trick

#130

Post by xtiansimon »

> “…LLM vendors [are responsible for the message?] We should be afraid […] The purpose here is not to responsibly warn us of a real threat. If that were the aim there would be a lot more shutting down of data centres…”

Let’s not forget these innovations are on the heels of COVID. Strong, swift action by government, industry, and individuals against a deadly pathogen is “controversial”. Even if killer AI was here, twice shy…

I’m angry about a lot of things right now, but LLM “marketing” (and inadequate reporting which turns to science fiction instead of science) is not one of them. The LLM revolution is getting shoehorned into this Three Card Monte narrative, and I don’t see the utility.

The criticisms of LLM promise and danger is part of the zeitgeist. If firms are playing off of anything I bet it’s that, and not an industry wide conspiracy to trick the public and customers. Advertising and marketing meets people where they’re at, and “imagines” where they want to go, all wrapped up with the product. It doesn’t make the product frightening. It’s the same for all manner of dangerous technologies—guns, nuclear energy, whatever. The product is the solution to the fear.

> “The LLMs we have today are famously obsequious. The phrase “you’re absolutely right!” may never again be used in earnest.”

Hard NO. I get it, the language patterns of LLMs are creepy, but it’s not bad usage. So, no.

I can handle the cognitive dissonance of computer algorithms spewing out anthropomorphic phrasing and not decide that I, as a human being, can no longer in humility and honesty tell someone else they’re right, and i was wrong.

Post reply on HN