ML promises to be profoundly weird • Hacker News Board

Re: ML promises to be profoundly weird

#561

Post by paganel » Thu, Apr 09, 2026, 1:50 PM UTC

eru wrote: post #518

paganel wrote: post #511

Earlier quoted context omitted.

> The general point is accurate, don’t take it so literally. It's not, because the Malthusian trap was all too real going into modernity, as in recurring famines were a thing, they were quite real, nothing "literal" about them.

Compare https://fass.nus.edu.sg/ecs/wp-content/uploads/sites/4/2020/...

First of all, the study is written by an economist, might as well have sent me an Oracle of Delphi pronouncement. And second, he mentions the Malthusian trap being a real thing in his very first sentence, so not sure what I should have gotten out of this.

Re: ML promises to be profoundly weird

#562

Post by steveklabnik » Thu, Apr 09, 2026, 1:56 PM UTC

otterley wrote: post #457

steveklabnik wrote: post #361

Earlier quoted context omitted.

Even beyond that, the initial legal opinion we do have did in fact point to training being fair use: https://www.reuters.com/legal/litigation/anthropic-wins-key-... However, I don't feel comfortable suggesting that this is settled just yet, one district judge's opinion does not mean that other future cases may disagree, or we may at some point get explicit legislation one way or the other.

I think the court dropped the ball here. On the one hand, I think they were right that using existing works--copyrighted or otherwise--to train a model was transformable fair use. On the other hand, Anthropic and others trained their models on illicit copies of the works; they (more often than not) didn't pay the copyright holders. There's a doctrine in Fifth Amendment law called "fruit of the poisonous tree." The ge…

I mean, that is what the court said! Training on pirated data was not fair use. Training on legally acquired data is fair use.

Anthropic legally acquired the data and re-trained on it before release.

Re: ML promises to be profoundly weird

#563

Post by famouswaffles » Thu, Apr 09, 2026, 2:06 PM UTC

kgeist wrote: post #513

famouswaffles wrote: post #419

Earlier quoted context omitted.

>And you know what, Qwen3's entire forward pass is just 50 lines of very simple code (mostly vector-matrix multiplications). The code being simple doesn't mean much when all the complexity is encoded in billions of learned weights. The forward pass is just the execution mechanism. Conflating its brevity with simplicity of the underlying computation is a basic misunderstanding of what a forward pass actually is. What…

>The code being simple doesn't mean much when all the complexity is encoded in billions of learned weights. The forward pass is just the execution mechanism. Conflating its brevity with simplicity of the underlying computation is a basic misunderstanding of what a forward pass actually is. What you've just said is the equivalent of saying blackbox.py is simple because 'python blackbox.py' only took 1 line. It's just…

I get irritated when people act like they know what they are talking about but then it's just nonsense they keep spitting out. I'm honestly sick of it. There's a fair amount of LLM interpretability research out there. If you're actually interested in knowing better then go read them. I'll even link what i find interesting. All this talk of lookup tables is nonsensical. You have no idea what you're talking about.

>It doesn't even "know" what the actual text continuation must be, strictly speaking. It just returns a list of probabilities that we must select. It can't select it itself. To go from "list of probabilities" to "chatbot" requires adding additional hardcoded code (no AI involved) that greatly influences how the chatbot behaves, feels. Imagine if an actual sentient being had a button: you press it, and suddenly Steven the sailor becomes a Chinese lady who discusses Confucius. Or starts saying random gibberish. There's no independent agency whatsoever. It's all a bunch of clever tricks.

You are not making any sense here. Producing a probability distribution over next tokens is the model’s decision procedure. Sampling is just the readout rule for turning that distribution into a concrete sequence. Yes, decoding choices affect style, creativity, determinism, and failure modes. That is true. It does not follow that the model is therefore “just tricks” or that the intelligence-like behavior lives outside the network.

>In an actual brain, the structure of the connectome itself drives a lot of behavior. In an LLM, all connections are static and predefined. A brain is much more resistant to failure. In an LLM changing a single hypersensitive neuron can lead to a full model collapse. There are humans who live normal lives with a full hemisphere removed.

You are moving goalposts. Fact is: randomly corrupting a system damages it. This is not a meaningful test of whether a system is "truly intelligent." Random lesions to human cortex are also catastrophic. The hemispherectomy cases you mention involve surgical removal of diseased tissue with significant neural reorganization over time, not random weight corruption. That's not even a fair comparison.

LLMs are also deeply redundant. If they weren't, techniques like quantization or layer pruning wouldn't work.

Re: ML promises to be profoundly weird

#564

Post by throwaway27448 » Thu, Apr 09, 2026, 2:10 PM UTC

simianwords wrote: post #257

throwaway27448 wrote: post #75

Earlier quoted context omitted.

> LLMs with harnesses are clearly capable of engaging with logical problems that only need text. To some extent. It's not clear where specifically the boundaries are, but it seems to fail to approach problems in ways that aren't embedded in the training set. I certainly would not put money on it solving an arbitrary logical problem.

> To some extent. It's not clear where specifically the boundaries are, but it seems to fail to approach problems in ways that aren't embedded in the training set. I certainly would not put money on it solving an arbitrary logical problem. In what way can you falsify this without having the LLM be omniscient? We have examples of it solving things that are not in the training set - it found vulnerabilities in 25 year…

I'm just saying I'd rather hire a human that can be reasoned with than rely on software that can't be. At least where reasoning is involved.

Granted, I don't do a lot of needle-in-the-haystack work like finding vulnerabilities where search will naturally dominate.

Also, I imagine most reasoning involved in exploits will be found in the training sets—there are only so many patterns of exploitation found in formal languages.

Re: ML promises to be profoundly weird

#565

Post by cmrdporcupine » Thu, Apr 09, 2026, 2:11 PM UTC

Retric wrote: post #370

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. This is just wildly incorrect. People started running out of trees during the early Iron Age. Woodlands have been a managed and often over exploited resource for a long time. Active agriculture vs passive woodlands vs animal grazing has been in constant tension for thousands…

This, and going back further, people literally would brutally massacre neighbouring tribal groupings over control of fishing and hunting and gathering grounds.

The rapid dispersal of our species over literally the entire planet (minus Antarctica) likely also has a lot to do with constantly moving on to new opportunities further away from rivals.

That said, starting in about the 18th century we ran out of new places for that. And intensification truly began.

Re: ML promises to be profoundly weird

#566

Post by philbitt » Thu, Apr 09, 2026, 2:19 PM UTC

[dead]

Re: ML promises to be profoundly weird

#567

Post by 0rbiter » Thu, Apr 09, 2026, 2:23 PM UTC

drob518 wrote: post #94

> It remains unclear whether continuing to throw vast quantities of silicon and ever-bigger corpuses at the current generation of models will lead to human-equivalent capabilities. Massive increases in training costs and parameter count seem to be yielding diminishing returns. Or maybe this effect is illusory. Mysteries! I’m not even sure whether this is possible. The current corpus used for training includes virtual…

[dead]

Re: ML promises to be profoundly weird

#568

Post by simianwords » Thu, Apr 09, 2026, 2:36 PM UTC

camgunz wrote: post #532

simianwords wrote: post #515

Earlier quoted context omitted.

the latest top reported agentic LLMs score about 83–87%, versus an original human baseline of about 25.3% end to end, so today’s best systems appear to outperform humans by roughly 58–62 percentage points, or about 3.3–3.4× So according to your own benchmark LLMs hallucinate much less than humans and report way higher accuracy. Do you agree to be more skeptical of humans than LLMs on these tasks?

1. Irrelevant. I've delivered example after example of your fave model bullshitting. You should've bitten the bullet long ago. Honestly I'm disappointed; I've seen you in a lot of AI threads and assumed you'd be good to talk to on this, but you've moved the goalposts over and over again rather than engage in good faith. Anyone reading this thread (god bless them) can see you're plainly not objective here, thus callin…

This was my original point

>I don't think calling AI a bullshit machine is correct. In spirit.

That was always my goal post and I asked the challenge to get it to bullshit to drive a point across. You yourself said it is trivial.

1. You came up with the horns question - I tried with the thinking model and it clearly understood that it was a joke and replied appropriately

2. You came up with the assembly question - I tried it again with the thinking model and it gave the right answer again

3. Now you gave up trying to make prompts by yourself because you realised that its in fact not trivial

4. Then you started looking for benchmarks to show that it bullshits

5. You picked a benchmark that doesn't allow tools (which was not my constraint)

6. Then you picked a benchmark that does allow tools, and it turns out that it performs much better than humans

7. Upon hearing this, you shifted to goal posts to say that "models don't know how to say I don't know and I can teach models etc etc"

On the last part: There's a benchmark called SimpleQA which doesn't allow tools and allows for "I don't know" as an answer and GPT 5 still beats humans.

I think you should reconsider thinking this "I don't think calling AI a bullshit machine is correct".

Re: ML promises to be profoundly weird

#569

Post by altruios » Thu, Apr 09, 2026, 2:39 PM UTC

lelanthran wrote: post #528

altruios wrote: post #388

Earlier quoted context omitted.

> I think you are walking all around the word "consent" and trying very hard to avoid it altogether. > Your perspective, because it refuses to include any sort of consent, is invalid. No perspective that refuses consent can be valid. This is what I was responding to. I do not understand your thinking in this post.

> This is what I was responding to. I do not understand your thinking in this post. I thought it was clear from "refuses to include any sort of consent" that I am talking specifically about holding an opinion that refuses to include consideration for consent, not refuses consent for usage.

But that's what I'm confused about:

How is freely giving consent for (all) others to read your content not 'considering consent'?

I'm not trying to be snarky. I really don't see the missing piece that isn't written that connects those dots.

Re: ML promises to be profoundly weird

#570

Post by 52-6F-62 » Thu, Apr 09, 2026, 2:45 PM UTC

LogicFailsMe wrote: post #138

52-6F-62 wrote: post #109

Earlier quoted context omitted.

> The Vogon constructor fleet is way overdue in my book Don't you see it? That's exactly what "AI" in this context is . It's the bypass. Where does it end, eh? Build a quantum "AI" that will end up just needing more data, more input. The end goal must starts looking like creating an entirely new universe, a complete clone of everything we have here so it can run all the necessary computations and we can... ? (You are…

You have absolutely no sense of perspective. We are all metabolically expensive meat machines whose only value is to propagate our genetic money shot. That we get to briefly entertain ourselves with consciousness and culture is IMO likely a mystery we will never solve without upgrading to running in a substrate more advanced than the MVP for sentience we currently pilot. Will we get there or will we wipe ourselves ou…

> You have absolutely no sense of perspective.

> metabolically expensive meat machines

Calm down. There is no meat. It's just a giant sponge of wave packets.

Your take is the most S.V. conformist one there is: materialist reductionism, and it is doomed.

Tell me you don't understand hitchhikers guide without telling me you don't understand hitchhikers guide. This is as bad as Thiel and ilk's inverted reading of LOTR.