ML promises to be profoundly weird • Hacker News Board

Re: ML promises to be profoundly weird

#281

Post by bigyabai » Wed, Apr 08, 2026, 7:47 PM UTC

munificent wrote: post #173

steveklabnik wrote: post #165

Earlier quoted context omitted.

As you know, I deeply respect you. Not trying to argue here, just provide my own perspective: > Why would a writer put an article online if ChatGPT will slurp it up and regurgitate it back to users without anyone ever even finding the original article? I write things for two main reasons: I feel like I have to. I need to create things. On some level, I would write stuff down even if nobody reads it (and I do do that…

Agreed, totally! I still write and put stuff online. But it definitely feels different now. It used to feel like I was tending a public garden filled with other people who might enjoy it. It still kind of feels like that, but there are a handful of giant combine machines grinding their way around the garden harvesting stuff and making billionaires richer at the same time. It's not enough to dissuade me from contribut…

If raw resources (tree cutting) and manufacturing (book binding) is saturated, a fully-realized economy has just one step left: financialization.

You have to start finding ways to keep people hooked on books and make it a part of their regular lifestyle. One book can't be enough, and after a while you have to convince them to replace the books they already bought. New editions, Author's Footnotes, limited run release, all of the stops have to be pulled out to get consumers to show up en-masse. Because that's what they are - consumers, not readers - wallets to be squeezed until they're bled of all the trust they had in media.

I think about the publications I liked reading as a kid, like Joystiq and Polygon. Some of the best games journalism the industry produced, but inevitably doomed to fail as their competitors monetized further. The rest of traditional media has followed the same path, converging on some mercurial social network marketing tactic as the placeholder for big-picture brand strategy.

Re: ML promises to be profoundly weird

#282

Post by ACCount37 » Wed, Apr 08, 2026, 7:47 PM UTC

Unearned5161 wrote: post #271

Articles like this should approach topics on consciousness with more humility than is displayed here. We don’t even agree on a good definition of what’s going on inside our own heads yet, what gives you the confidence to say that what goes on inside an LLM can’t be conscious?

Obviously, the LLMs lack the divine spark, so they can't be conscious. Same as clones, IVF babies, or half of all the twins.

Jest aside, I do agree. If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is.

Re: ML promises to be profoundly weird

#283

Post by rafterydj » Wed, Apr 08, 2026, 7:47 PM UTC

steveklabnik wrote: post #187

munificent wrote: post #173

Earlier quoted context omitted.

Agreed, totally! I still write and put stuff online. But it definitely feels different now. It used to feel like I was tending a public garden filled with other people who might enjoy it. It still kind of feels like that, but there are a handful of giant combine machines grinding their way around the garden harvesting stuff and making billionaires richer at the same time. It's not enough to dissuade me from contribut…

I can totally see that, for sure. I was much more likely to write a review long ago, now I don't even bother. (For buying stuff online, at least.) Maybe I lost my innocence about this stuff a long time ago, and so it's not so much LLMs that broke it for me, but maybe... I dunno, the downfall of Web 2.0 and the death of RSS? I do think that the old internet, for some definition of "old," felt different. For sure. I'll…

I feel the future includes the sentiments you describe. It was a little before my time professionally, but I grew up reading that kind of thinking.

I do think that the open web stuff, decentralized, or at least more decentralized than currently, is the path forward. I've been reading about the AT protocol and it recently becoming an official working group with the IETF.

I feel a second order effect of making decentralized social networking easier, is making individuals more empowered to separate from what they don't believe in. The third order effect is then building separate infrastructure entirely.

As sad as that can be - in my personal opinion it runs the risk of ending the "world wide" part of the web - it appears to be the only way society can avoid enriching the few beyond reason.

Re: ML promises to be profoundly weird

#284

Post by simianwords » Wed, Apr 08, 2026, 7:48 PM UTC

drob518 wrote: post #168

danieltanfh95 wrote: post #71

I think the discussion has to be more nuanced than this. "LLMs still can't do X so it's an idiot" is a bad line of thought. LLMs with harnesses are clearly capable of engaging with logical problems that only need text. LLMs are not there yet with images, but we are improving with UI and access to tools like figma. LLMs are clearly unable to propose new, creative solutions for problems it has never seen before.

> "LLMs still can't do X so it's an idiot" Let’s be careful. That’s a straw man. I don’t know anyone who says that. Aphyr says in the article that AIs can do things. But they have been marketed as “intelligent,” and I agree with Aphyr that the word is suggesting way more than AIs currently deliver. They do not reason and they do not think and are not truly intelligent. As the article says, they are big wads of linear…

> They do not reason

How do you disprove it?

Re: ML promises to be profoundly weird

#285

Post by simoncion » Wed, Apr 08, 2026, 7:49 PM UTC

simianwords wrote: post #259

simoncion wrote: post #253

Earlier quoted context omitted.

Neat. So, despite the fact that it looks like you have to pay for ChatGPT Voice mode with video, [0] it doesn't count as an example of it bullshitting on ChatGPT (paid version) That is, father_phi's use of what seems to be a paid version of ChatGPT to have a bullshit-filled conversation that definitely spans less than four pages doesn't count? [0] The page at [1] declares that the video feature is "Available in ChatG…

Lets stick to my challenge please - thinking version, find bullshit. If you can't, thats ok. Do you accept then under the constraints that the thinking version doesn't produce bullshit?

Given aphyr's vocation (and how very lucrative it is), and how years and years of his writing indicates that he's very devoted to getting a correct and complete answer when investigating a question, I find it hard to believe that he's not using a paid version of the LLMs. If I knew him, I'd ask and verify, but I don't, so I won't.

> Lets stick to my challenge please...

I did. Your challenge was literally:

  If it bullshits so much, you wouldn't have a problem giving me an example of it bullshitting on ChatGPT (paid version)? Lets take any example of a text prompt fitting a few pages - it may be a question in science or math or any domain. Can you get it to bullshit?

father_phi's two-sentence question about the whether one can use a cup that's closed at the top and open at the bottom definitely counts. Given what I've mentioned about apyhr above, I expect he has already run your challenge on the fanciest-available version and reported on the results in the essay under discussion.

Re: ML promises to be profoundly weird

#286

Post by simianwords » Wed, Apr 08, 2026, 7:50 PM UTC

I so far asked few people to make GPT-5.4 thinking to bullshit (with max 4 pages of prompt), no one can find an example.

But the way people speak in general, as well as this post, implies that such a challenge can easily be beaten. If so, I'm not able to find examples.

Re: ML promises to be profoundly weird

#287

Post by ACCount37 » Wed, Apr 08, 2026, 7:51 PM UTC

hedgehog wrote: post #145

rudhdb773b wrote: post #120

Earlier quoted context omitted.

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).

If platonic representation hypothesis holds across substrates, then it might matter very little, in the end. It holds across architectures in ML, empirically.

The crowd of "backpropagation and Hebbian learning + predictive coding are two facets of the very same gradient descent" also has a surprisingly good track record so far.

Re: ML promises to be profoundly weird

#288

Post by ACCount37 » Wed, Apr 08, 2026, 7:52 PM UTC

hedgehog wrote: post #258

rudhdb773b wrote: post #182

Earlier quoted context omitted.

Sure the implementation details are different. I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting? And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?

Given the large visible differences in behavior and construction, akin to the difference between a horse and a pickup truck, I would ask the reverse question: In what ways do LLMs meet the definition of having consciousness and agency? Veering into the realm of conjecture and opinion, I tend to think a 1:1 computer simulation of human cognition is possible, and transformers being computationally universal are thus th…

What about modern LLMs isn't "agentic" enough?

Doesn't matter if they're conscious for that. They're clearly capable of goal oriented behavior.

Re: ML promises to be profoundly weird

#289

Post by computably » Wed, Apr 08, 2026, 7:52 PM UTC

steveklabnik wrote: post #165

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

As you know, I deeply respect you. Not trying to argue here, just provide my own perspective: > Why would a writer put an article online if ChatGPT will slurp it up and regurgitate it back to users without anyone ever even finding the original article? I write things for two main reasons: I feel like I have to. I need to create things. On some level, I would write stuff down even if nobody reads it (and I do do that…

> A lot of people read things, it changes their life, and their life is better. They may not even remember where they read these things. They don't produce citations all of the time. That's totally fine, and normal. I don't see LLMs as being any different. If I write an article about making code better, and ChatGPT trains on it, and someone, somewhere, needs help, and ChatGPT helps them? Win, as far as I'm concerned. Even if I never know that it's happened. I already do not hear from every single person who reads my writing.

Not a contradiction but an addendum: plenty of creative pursuits are not about functional value, or at least not primarily. If somebody writes a seemingly genuine blog post about their family trauma, and I as the reader find out it's made-up bullshit, that's abhorrent to me, whether or not AI is involved. And I think it would be perfectly fair for writers who do create similar but genuine content to find it abhorrent that they must compete with genAI, that genAI will slurp up their words, and that genAI's mere existence casts doubt on their own authenticity. It's not about money or social utility, it's about human connection.

Re: ML promises to be profoundly weird

#290

Post by jsmo » Wed, Apr 08, 2026, 7:53 PM UTC

konschubert wrote: post #219

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> Prior to the industrial revolution, the natural world was nearly infinitely abundant. The opposite is true. Central Europe was almost devoid of trees. Food was scarce as arable land bore little fruit without fertiliser. Society was Malthusian until the Industrial Revolution.

Can we interpret "abundant" in a Darwinian sense e.g. diversity of life? I would think the industrial farming revolution decreased crop variety over time same for animal lineages aside from the rapid increase in mixed poodle breeds.