Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

431–440 of 641 posts

Re: ML promises to be profoundly weird

#431

Post by navaed01 »

munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

The natural world was not meaningfully abundant… Way before the industrial revolution land which was once used for opening hunting was closed off by the ruling class. Even before the Industrial Revolution you had a new class of merchant and factory owners who earned riches to buy land and keep the poor from hunting on it. Much of the natural resources out of reach for the majority and only accessible by those with deep pockets

Re: ML promises to be profoundly weird

#432

Post by dTal »

squigz wrote: post #404
Quarrelsome wrote: post #403

Earlier quoted context omitted.

we're talking about the fucking industrial revolution, of course this defaults to the European perspective. Unless you wanna spit some new bars about Aztec foundries and train lines connecting meso-america in the 19th century, then the point stands. At that time, the world appeared to the industrialists of the industrial revolution to be infinite. Nor had humanity discovered the terrible side effects of fossil fuels…

Sure, of course it's convenient to ignore the native peoples and pretend that prior to the Industrial Revolution the rest of the world outside of Europe was some untapped well of resources that Europeans had a natural right to. Who might be swept underfoot in this "Information Revolution", I wonder?

Nobody said anything about Europeans having a "natural right". Bad enough to derail a conversation with irrelevant political nitpicking, unforgiveable to use a strawman to do so. Boo.

Re: ML promises to be profoundly weird

#433

Post by saghm »

mkehrt wrote: post #265
beders wrote: post #79

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency and that the term AI is just completely overloaded right now.

One thing that has happened is that "AI" has been an academic discipline since literally the 1950s. The term was originally used in the hope that we would soon be able to emulate human minds. This turned out to be hard, but the name stuck to the discipline. Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like.…

> Now, suddenly, this name has been broadcast to every human in the world more or less. To them, it's a new term, and it obviously means something human mind-like. But to people who work on AI, that's not generally what it means. (Which isn't to say that some of them don't think we're near to achieving that; they just use other terms like "AGI" for that goal). So the name, which has a long history, is deceptive to people who aren't familiar with computer science.

I think it's even worse than that: people were familiar with the term already, but from science fiction, where it referred to actually human-level intelligence. It's similar to the "hoverboard" thing from a while back, except this time with profoundly higher stakes and requires for more technical knowledge to be able to see that it is in fact touching the ground.

Re: ML promises to be profoundly weird

#434

Post by navaed01 »

munificent wrote: post #173
steveklabnik wrote: post #165

Earlier quoted context omitted.

As you know, I deeply respect you. Not trying to argue here, just provide my own perspective: > Why would a writer put an article online if ChatGPT will slurp it up and regurgitate it back to users without anyone ever even finding the original article? I write things for two main reasons: I feel like I have to. I need to create things. On some level, I would write stuff down even if nobody reads it (and I do do that…

Agreed, totally! I still write and put stuff online. But it definitely feels different now. It used to feel like I was tending a public garden filled with other people who might enjoy it. It still kind of feels like that, but there are a handful of giant combine machines grinding their way around the garden harvesting stuff and making billionaires richer at the same time. It's not enough to dissuade me from contribut…

It does feel like the collaborative, free open nature of the web has gone and the optimism that brought… it feels like no one would build Foursquare today. But then I wonder if I’m just old an jaded and to the younger generation creating content, for them the web is open and expressive- just in a different way

Re: ML promises to be profoundly weird

#435

Post by joefourier »

famouswaffles wrote: post #424
joefourier wrote: post #418

Earlier quoted context omitted.

> That's not true. Modern training techniques aren't enough. Vanilla RNNs with modern training techniques still scale poorly. You have to make some pretty big architectural divergences (throwing away recurrency during training) to get a RNN to scale well. SSMs move the non-linearity outside of the recurrence which enables parallelisation during training. It is trivial to do this architectural change with an LSTM (see…

>SSMs move the non-linearity outside of the recurrence which enables parallelisation during training. It is trivial to do this architectural change with an LSTM (see the xLSTM paper). Linear RNNs are still RNNs. Removing the non-linearity from the recurrence path is exactly what constitutes a "pretty big architectural divergence." A linear RNN is an RNN in a structural sense, certainly, but functionally it strips out…

> That does not scale anywhere near as well as Transformers in compute spend. It's paper/research novelty. Nobody will be doing this for production.

What exactly makes you so confident?

The world is not just labs that can afford billion dollar datacentres and selling access to SOTA LLMs at $30/Mtokens. Transformers are highly unsuitable for many applications for a variety of reasons and non-linear RNNs trained via parallel methods are an extremely attractive value proposition and will likely feature in production in the next products I work on.

> I guess there's some misunderstanding here because Qwen is 100% a transformer, not a hybrid RNN/LSTM whatever.

See the Qwen3.5 Huggingface description: https://huggingface.co/Qwen/Qwen3.5-27B > Efficient Hybrid Architecture: Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.

Re: ML promises to be profoundly weird

#436

Post by saghm »

erichocean wrote: post #119
beders wrote: post #79

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency and that the term AI is just completely overloaded right now.

AI is exactly the right term: the machines can do "intelligence", and they do so artificially. Just like we have machines that can do "math", and they do so artificially. Or "logic", and they do so artificially. I assume we'll drop the "artificial" part in my lifetime, since there's nothing truly artificial about it (just like math and logic), since it's really just mechanical. No one cares that transistors can do ma…

> Just like we have machines that can do "math", and they do so artificially.

Nobody calls calculators "artificial mathemeticians", though; we refer to them by a unique word that defines what they can and can't do in a far less fanciful and ambiguous way.

Re: ML promises to be profoundly weird

#437

Post by navaed01 »

idle_zealot wrote: post #382
munificent wrote: post #299

Earlier quoted context omitted.

> I'm from the "information wants to be free" sort of persuasion, and now that largely makes me feel kinda old. Me too, 100%. But that was during a moment in time when that information was more likely to be enabling a person who otherwise didn't have as many resources than enabling a billionaire to make their torment nexus 0.1% more powerful. > I kinda had to accept that stuff I make was going to be used towards ends…

> But in a world with AI, I worry that anything I put out there increases inequality and that gives me the heebie-jeebies. Maybe that's just the way things are now and I have to accept it. This observation doesn't really clash with "information wants to be free." You just have to include LLMs in the category or "information," like Free Software types already do for all software. You don't need to abandon your princip…

I don’t disagree with you, but this has been going on for a while… Google monetized the the by indexing it and monetized what you wanted to find. Facebook monetized the eyeballs from the pictures and posts you added. Now LLMs will monetize all web content. To play devil’s advocate - LLMs do give something back. Those with ideas and no coding experience can now build entire businesses for little to zero cost. This seems different

Re: ML promises to be profoundly weird

#438

Post by jtbaker »

Retric wrote: post #370
munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

> Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. This is just wildly incorrect. People started running out of trees during the early Iron Age. Woodlands have been a managed and often over exploited resource for a long time. Active agriculture vs passive woodlands vs animal grazing has been in constant tension for thousands…

the Stepchange show went fairly deep on this topic in their first episode (listened to it recently). https://www.stepchange.show/coal-part-i

Re: ML promises to be profoundly weird

#439

Post by munificent »

elros wrote: post #410
ryandrake wrote: post #250

Earlier quoted context omitted.

> But as Amazon got bigger and bigger and the externalities more apparent, it felt less like we were helping each other and more like we were help Bezos buy yet another yacht or media empire. The Internet-circulating quote comes to mind: Planet Earth is pretty much a vacation resort for around 500 rich people, and the remaining 8 billion of us are just their staff. The Relative Few have got the system set up perfectl…

> The Internet-circulating quote comes to mind: Planet Earth is pretty much a vacation resort for around 500 rich people, and the remaining 8 billion of us are just their staff. The Relative Few have got the system set up perfectly so that whatever we do, we're probably serving/enriching them. AI doesn't really change this, but it does further it. I don't necessarily disagree with the analysis on how Planet Earth is…

I think you're right.

Anyone who finds themselves with $100m in their bank account and thinks, "No, I need more," is a person with a hole inside them that can never be filled.

Re: ML promises to be profoundly weird

#440

Post by munificent »

idle_zealot wrote: post #382
munificent wrote: post #299

Earlier quoted context omitted.

> I'm from the "information wants to be free" sort of persuasion, and now that largely makes me feel kinda old. Me too, 100%. But that was during a moment in time when that information was more likely to be enabling a person who otherwise didn't have as many resources than enabling a billionaire to make their torment nexus 0.1% more powerful. > I kinda had to accept that stuff I make was going to be used towards ends…

> But in a world with AI, I worry that anything I put out there increases inequality and that gives me the heebie-jeebies. Maybe that's just the way things are now and I have to accept it. This observation doesn't really clash with "information wants to be free." You just have to include LLMs in the category or "information," like Free Software types already do for all software. You don't need to abandon your princip…

> A handful of companies can't be allowed to benefit from free information and then put what they make behind a wall.

What is there to prevent them?

Post reply on HN