Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

411–420 of 641 posts

Re: ML promises to be profoundly weird

#411

Post by irishcoffee »

nick32661123 wrote: post #309
munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

Our only hope is that AI in the long run is both powerful and benevolent enough to be its own "whistleblower" in cases of misuse.

I struggle so hard with this anthropomorphism of LLMs. At the end of the day it's a statistical gradient descent predictor with a bunch of "shit" bolted on top to try and steer outputs in a specific way.

They don't have the actual concept of "benevolent"... or a concept of anything at all. Based on an input, they regress down a path of "what is the next most probable statistical token to output next" and that's fucking it, with the bolted-on shit manipulating these outputs a bit.

I don't doubt that at some point there will be some other AI leap, but I'm not even sure it'll be built on this foundation.

What really needs to be developed is an actual artificial brain of sorts. Much like an infant learns language from first principals, a real AI would have a phase of continuous growth, creating actual memories and being able to reflect upon them. I daresay context windows are not that.

I'd really like to encourage anyone to pump the brakes a bit on how these things actually work, and what they actually are. There is a reason sama is pivoting away from video, et. al. and into corporate software coding, much like anthropic.

Re: ML promises to be profoundly weird

#412

Post by felipeerias »

Retric wrote: post #370
munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. This is just wildly incorrect. People started running out of trees during the early Iron Age. Woodlands have been a managed and often over exploited resource for a long time. Active agriculture vs passive woodlands vs animal grazing has been in constant tension for thousands…

People had been hunting whales for centuries, but industrialisation gave them the means and the motivation to do so until near extinction.

Re: ML promises to be profoundly weird

#413

Post by echion »

idle_zealot wrote: post #382
munificent wrote: post #299

Earlier quoted context omitted.

> I'm from the "information wants to be free" sort of persuasion, and now that largely makes me feel kinda old. Me too, 100%. But that was during a moment in time when that information was more likely to be enabling a person who otherwise didn't have as many resources than enabling a billionaire to make their torment nexus 0.1% more powerful. > I kinda had to accept that stuff I make was going to be used towards ends…

> But in a world with AI, I worry that anything I put out there increases inequality and that gives me the heebie-jeebies. Maybe that's just the way things are now and I have to accept it. This observation doesn't really clash with "information wants to be free." You just have to include LLMs in the category or "information," like Free Software types already do for all software. You don't need to abandon your princip…

> Free Software types already do for all software

Free Software types also create software...they didn't just argue for a better license and try to regulate Sun/others to re-license their software; they wrote free (libre) versions of proprietary software and released it for free (cost), which is what counteracted the "[putting] what they make behind a wall". If you're saying "[some] LLMs should be free", I agree.

Re: ML promises to be profoundly weird

#414

Post by some_random »

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

That is straightforwardly not true, land ownership was very well defined and the people who hunted in it without permission were prosecuted.

Re: ML promises to be profoundly weird

#415

Post by derangedHorse »

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> If all of us can go hunting in the woods and yet there is still game to be found, then there's no compelling reason to define and litigate who "owns" those woods.

Property rights don't just protect natural resources, but labor as well. If I cleared out hunting ground in that forest to be the prime spot to catch animals, I would make sure I can use it when I want.

> a small number of people were able to completely deplete parts of the earth

A small number of people seems inaccurate when there's typically many more individuals in the pipelines for these technologies.

> and in return profit off the knowledge over and over again at industrial scale

Not off just that knowledge, there needed to be a model trained on the data of many others to utilize it.

> Why would a writer put an article online if ChatGPT will slurp it up and regurgitate it back to users without anyone ever even finding the original article?

Who's better at writing in this scenario and what are my motivations? If it's ChatGPT and I did it for money, then I would say I should recognize that I can't compete and find something AI can't do. If it's ChatGPT and I write to convey my ideas in an effort to learn regardless of the bestowment of a new perspective on the reader, I'll keep writing.

> Why would anyone plant seeds on someone else's farm?

They wouldn't unless it was their own way to attain food and survive. And if it's not the only way, they can defer to those with optimal methods to get it the cheapest they can in the market.

Re: ML promises to be profoundly weird

#417

Post by AlexCoventry »

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> If all of us can go hunting in the woods and yet there is still game to be found, then there's no compelling reason to define and litigate who "owns" those woods.

https://en.wikipedia.org/wiki/Feudalism

Re: ML promises to be profoundly weird

#418

Post by joefourier »

famouswaffles wrote: post #407
joefourier wrote: post #334

Earlier quoted context omitted.

With modern training techniques, RNNs (not just linear SSMs, potentially even vanilla LSTMs) can scale just as well as transformers or even better when it comes to enormous context lengths. Dot-product attention has better performance in a number of domains however (especially for exact retrieval) so the best architectures are likely to remain hybrid for now.

>With modern training techniques, RNNs (not just linear SSMs, potentially even vanilla LSTMs) can scale just as well as transformers or even better when it comes to enormous context lengths. That's not true. Modern training techniques aren't enough. Vanilla RNNs with modern training techniques still scale poorly. You have to make some pretty big architectural divergences (throwing away recurrency during training) to…

> That's not true. Modern training techniques aren't enough. Vanilla RNNs with modern training techniques still scale poorly. You have to make some pretty big architectural divergences (throwing away recurrency during training) to get a RNN to scale well.

SSMs move the non-linearity outside of the recurrence which enables parallelisation during training. It is trivial to do this architectural change with an LSTM (see the xLSTM paper). Linear RNNs are still RNNs.

But you can still keep the non linearity by training with parallel Newtown methods, which work on vanilla LSTMs and scale to billion of parameters.

> None of the big labs seem to be bothered with hybrid approaches.

Does Alibaba not count? Qwen3.5 models are the top performers in terms of small models as far as my tests and online benchmarks go.

Re: ML promises to be profoundly weird

#419

Post by famouswaffles »

kgeist wrote: post #366
_dwt wrote: post #58

I have a question for all the "humans make those mistakes too" people in this thread, and elsewhere: have you ever read, or at least skimmed a summary of, "The Origin of Consciousness in the Breakdown of the Bicameral Mind"? Did you say "yeah, that sounds right"? Do you feel that your consciousness is primarily a linguistic phenomenon? I am not trying to be snarky; I used to think that intelligence was intrinsically…

>and while I agree humans can make similar mistakes/confabulations, I overwhelmingly feel that there is no "there" there. What really opened my eyes a couple weeks ago (anyone can try this): I asked Sonnet to write an inference engine for Qwen3, from scratch, without any dependencies, in pure C. I gave it GGUF specs for parsing (to quickly load existing models) and Qwen3's architecture description. The idea was to se…

>And you know what, Qwen3's entire forward pass is just 50 lines of very simple code (mostly vector-matrix multiplications).

The code being simple doesn't mean much when all the complexity is encoded in billions of learned weights. The forward pass is just the execution mechanism. Conflating its brevity with simplicity of the underlying computation is a basic misunderstanding of what a forward pass actually is. What you've just said is the equivalent of saying blackbox.py is simple because 'python blackbox.py' only took 1 line. It's just silly reasoning.

>After the pass, you need to choose the sampling strategy: how to choose the next token from the list. And this is where you can easily make the whole model much dumber, more creative, more robotic, make it collapse entirely by just choosing different decoding strategies. So a large part of a model's perceived performance/feel is not even in the neurons, but in some hardcoded manually-written function.

So ? I can pick the least likely token every time. The result would be garbage but that doesn't say anything about the model. The popular strategy is to randomly pick from the top n choices. What do you is keeping thousands of tokens coherent and on point even with this strategy ? Why don't you try sampling without a large language model to back it and see how well that goes for you ?

>Then I also performed "surgery" on this model by removing/corrupting layers and seeing what happens. If you do this excercise, you can see that it's not intelligence. It's just a text transformation algorithm. Something like "semantic template matcher". It generates output by finding, matching and combining several prelearned semantic templates. A slight perturbation in one neuron can break the "finding part" and it collapases entirely: it can't find the correct template to match and the whole illusion of intelligence breaks. Its corrupted output is what you expect from corrupting a pure text manipulation algorithm, not a truly intelligent system.

What do you think happens when you remove or corrupt arbitrary regions of the human brain? People can lose language, vision, memory, or reasoning, sometimes catastrophically.

Re: ML promises to be profoundly weird

#420

Post by School-Cotton »

drob518 wrote: post #395
simianwords wrote: post #284

Earlier quoted context omitted.

> They do not reason How do you disprove it?

We know that they do not reason because we know the algorithm behind the curtain. The model is generating the next token via model weights and some randomness. That’s all. It not reasoning. Sometimes it has an appearance of reasoning, but not if you know how it works. It doesn’t matter that the model manufacturer marketing department slaps a “Reasoning!” sticker on the side of the model. It’s not actually doing that.…

What is reasoning?
Post reply on HN