Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are a 400-year-long confidence trick

tomrenner.com

Post reply on HN
Print view View original

111–120 of 202 posts

Re: LLMs are a 400-year-long confidence trick

#111

Post by 112233 »

ManuelKiessling wrote: post #49
krystofee wrote: post #32

I disagree with the "confidence trick" framing completely. My belief in this tech isn't based on marketing hype or someone telling me it's good – it's based on cold reality of what I'm shipping daily. The productivity gains I'm seeing right now are unprecedented. Even a year ago this wouldn't have been possible, it really feels like an inflection point. I'm seeing legitimate 10x gains because I'm not writing code any…

This. By now I don’t understand how anyone can still argue in the abstract while it’s trivial to simply give it a try and collect cold, hard facts. It’s like arguing that the piano in the room is out of tune and not bothering to walk over to the piano and hit its keys.

So I tried it and it is worse that having random dude from Fiverr write you code — it is actively malicious and goes out of it's way do decieve and to subtly sabotage existing working code.

Do I now get the right to talk badly about all LLM coding, or is there another exercise I need to take?

Re: LLMs are a 400-year-long confidence trick

#112

Post by adrianN »

qsera wrote: post #100
adrianN wrote: post #22

Earlier quoted context omitted.

Intelligence is not magic though. The difference between intelligence and mathematics can plausibly be the same kind of difference between chemistry and intelligence.

There is Intelligence and there is Imitation of Intelligence. LLMs do the latter. Talk to any model about deep subjects. You ll understand what I am saying. After a while it will start going around in circles. FFS ask it to make an original joke, and be amused..

Many animals are clearly intelligent, but I can't talk to them at all.

Re: LLMs are a 400-year-long confidence trick

#113

Post by mossTechnician »

iNic wrote: post #94
mossTechnician wrote: post #27

Earlier quoted context omitted.

If it makes sense to handle all of these issues, then couldn't these organizations just acknowledge all of these issues? If reducing harm is the goal, I don't see a reason to totally segregate different issues, especially not by drawing a dividing line between the ones OpenAI already acknowledges and the ones it doesn't. I've never seen any self-described "AI safety" organizations that tackles any of the present-day…

If you've never seen it then you haven't been paying attention. For example Anthropic (the biggest AI org which is "safety" aligned) released a big report last year on metal well being [1]. Also here is their page on societal impacts [2]. Here is PauseAI's list of risks [3], it has deepfakes as its second issue! The problem is not that no one is trying to solve the issues that you mentioned, but that it is really har…

PauseAI's official proposal recommends[0]: "Only allow deployment of models after no dangerous capabilities are present." Their list of dangerous capabilities[1] does not include deepfakes, but it does include several unrealized ones that fit the description of this post here, including "a recursive loop of self-improvement, spinning rapidly out of control... called an intelligence explosion".

I appreciate you pointing out the Risks page though, as it does disprove my hyperbole about ignoring present-day harms completely, although I was disheartened that the page just appears to list things that they believe actions "could be mitigated by a Pause" (emphasis mine).

[0]: https://pauseai.info/proposal

[1]: https://pauseai.info/dangerous-capabilities

Re: LLMs are a 400-year-long confidence trick

#114

Post by kubb »

ACCount37 wrote: post #75
latexr wrote: post #51

Earlier quoted context omitted.

That is not necessarily true. That would be like arguing there is a finite number of improvements between the rockets of today and Star Trek ships. To get warp technology you can’t simply improve combustion engines, eventually you need to switch to something else. That could also apply to LLMs, that there would be a hard wall that the current approach can’t breach.

If that's the case, then, what's the wall? The "walls" that stopped AI decades ago stand no more. NLP and CSR were thought to be the "final bosses" of AI by many - until they fell to LLMs. There's no replacement. The closest thing to a "hard wall" LLMs have is probably online learning? And even that isn't really a hard wall. Because LLMs are good at in-context learning, which does many of the same things, and can do…

The wall is training data. Yes, we can make more and more of post training examples. No, we can never make enough. And there are diminishing returns to that process.

Re: LLMs are a 400-year-long confidence trick

#115

Post by falloutx »

ACCount37 wrote: post #99
falloutx wrote: post #77

Earlier quoted context omitted.

All of the professions its trying to replace are very much bottom end of the tree, like programmers, designers, artists, support, lawyers etc. While you can easily already replace management and execs with it already and save 50% of the costs, but no one is talking about that. At this point the "trick" is to scare white collar knowledge workers into submission with low pay and high workload with the assumption that A…

"Just replace management and execs with AI" is an elaborate wagie cope. "Management and execs" are quite resistant to today's AI automation - and mostly for technical reasons. The main reason being: even SOTA AIs of today are subhuman at highly agentic tasks and long-horizon tasks - which are exactly the kind of tasks the management has to handle. See: "AI plays Pokemon", AccountingBench, Vending-Bench and its "real…

Some management would be cut off when the time comes, Execs on the other hand are not there for work and are in due to personal relationships, so impossible to fire. If you think someone like lets say Satya Nadella can't be replaced by a bot which takes different input streams and then makes decisions, then you are joking. Even his recent end of 2025 letter was mostly written by AI.

Re: LLMs are a 400-year-long confidence trick

#116

Post by JacoboJacobi »

latexr wrote: post #98
JacoboJacobi wrote: post #86

Earlier quoted context omitted.

The claims being made that are cited are not really in that camp though.. It may be extremely dangerous to release. True. Even search engines had the potential to be deemed too dangerous in the nuclear pandoras box arguments of modern times. Then there are high-speed phishing opportunities, etc. It may be an essential failure to miss the boat. True. If calculators were upgraded/produced and disseminated at modern Int…

> The claims being made that are cited are not really in that camp though.. They literally are. Sam Altman has literally said multiple times this tech will cure cancer.

Such claims are not cited in the article. It may be possible to write a good article on the topic but this article could just as well be on the organic uptake of the PC and how most wealthy nontechnical people adopted a need for a PC on "cons" that preceded their ability to gain more worth than trouble.

Re: LLMs are a 400-year-long confidence trick

#117

Post by mpweiher »

krystofee wrote: post #32

I disagree with the "confidence trick" framing completely. My belief in this tech isn't based on marketing hype or someone telling me it's good – it's based on cold reality of what I'm shipping daily. The productivity gains I'm seeing right now are unprecedented. Even a year ago this wouldn't have been possible, it really feels like an inflection point. I'm seeing legitimate 10x gains because I'm not writing code any…

> I'm seeing legitimate 10x gains...

Self-reports on this have been remarkably unreliable.

Re: LLMs are a 400-year-long confidence trick

#118

Post by latexr »

petesergeant wrote: post #89

Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago. You may not like LLMs, you may not want LLMs, but pretending they're not doing anything clever or useful is bizarre, however flowery you make your language.

> Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago.

That’s quite a funny take, because I bet you someone will have made that same argument to criticise “crypto-deniers”.

> pretending they're not doing anything clever or useful

That isn’t at all the argument of the article. No one is claiming LLMs are completely useless or that they aren’t interesting technology. The critique is they’re being sold as way more than what they are or could be, that that has tangible negative consequences we can already feel, and the benefits don’t offset it.

Re: LLMs are a 400-year-long confidence trick

#119

Post by [deleted] »

latexr wrote: post #70
remus wrote: post #55

Earlier quoted context omitted.

The point of the article is to paint LLMs as a confidence trick, the keyword being trick. If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick? If a street performer was doing the cup and ball scam, but I actually won and left with more money than I started with then I'd say that's a pretty bad trick! Of course it is a little more nuanced than this and I would agree that s…

> If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick? Yes, yes you can. As I’ve mentioned elsewhere on this thread: > When a con man sells you a cheap watch for an high price, what you get is still useful—a watch that tells the time—but you were also still conned, because what you paid for is not what was advertised. You overpaid because you were tricked about what you w…

[deleted]

Re: LLMs are a 400-year-long confidence trick

#120

Post by kubb »

Al-Khwarizmi wrote: post #35
lyu07282 wrote: post #23

I think it's interesting how gamers have developed a pretty healthy aversion to generative ai in video games. Steam and Itch both now make it mandatory that games disclose generative ai use and recently even beloved Larian Studios was under fire for using ai for concept art. Gamers hate that shit. I think that's good, but the whole "AI is literally not doing anything", that it's just some mass hallucination has to di…

I haven't gamed much in the last few years due to severe lack of time so I'm out of touch, but I used to play a lot of CRPGs and I always dreamed of having NPCs who could talk and react beyond predefined scripted lines. This seems to finally be possible thanks to LLMs and I think it was desired by many (not only me). So why are gamers not excited about generative AI?

Because of how very badly it works in practice. LLM writing is bad, there’s no interactivity (inference delay), it feels soulless, there’s no coherence in the generated text.

There should have been one good game by now. But there isn’t.

Post reply on HN