Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I love LLMs, I hate hype

geohot.github.io

Post reply on HN
Print view View original

181–190 of 340 posts

Re: I love LLMs, I hate hype

#181

Post by int_19h »

jacobgold wrote: post #88
lukan wrote: post #70

Earlier quoted context omitted.

Sure, nothing is magic. You can go look how a simple LLM works and build your understanding from there. But calling it "just software" is trivializing it in my opinion. I can write software, but I cannot write software that writes software.

> But calling it "just software" is trivializing it in my opinion The bigger mistake would be trivializing the rest of the technology involved just because LLMs are the newest piece. LLMs are only "magic" because they're built on a stack that was already "magic" without them. LLMs are impossible without: - operating systems - programming languages - compilers - data centers / power grids / air conditioning - servers…

I would argue that LLMs are still magic, because, unlike the rest of this list, we still don't actually know how exactly they do what they do. We know how to build and train them, and we have general ideas about why e.g. attention is important, but if we actually knew how they tick we could do the same thing in a much smaller package (by handcoding everything).

Re: I love LLMs, I hate hype

#182

Post by int_19h »

Jtarii wrote: post #153
nylonstrung wrote: post #45

Earlier quoted context omitted.

Yeah but it was only like 2 years ago that artists were arguing this on the basis that AI-gen images would consistently mangle hands Now we're at a point where that never happens, and where lipsync is almost a completely solved problem If the issue here is simply that the quality is bad, one has to contend with the fact that it is undoubtedly exponentially improving and there's no reason we should expect that improve…

An LLM cannot make art because it isn't human. It can make "art like artifacts". Art involves one human communicating some emotional experience to another human, LLMs cannot experience human emotion, so they cannot make art. The process of making art is not a subset of hill climbing optimisation algorithms.

By this definition most of our culture isn't art because it's made not to "communicate some emotional experience" but to induce one in order to make money. 90% of pop music is like that, for example. And LLMs can do that just as well - better, probably, since they know all the psychological tricks from their training data.

Re: I love LLMs, I hate hype

#183

Post by wizzwizz4 »

fragmede wrote: post #180
simoncion wrote: post #140

Earlier quoted context omitted.

> This is like shoving a sponge down your windpipe to remove mucus. In my personal experience, not using a cotton-tipped swab for the task is like cleaning a plate loaded with gunk and burned-on patches with one's bare hands rather than choosing to use a sponge and/or brush. You can do it, [0] but it's much more work, much more time consuming, or you get an inferior result. [0] In my case, I'd need to make one set of…

We live in the future. You can get an ear cleaning camera endoscope device for $40 next day Amazon delivery anywhere they reach.

Or you could just take a shower, which makes it easy to wipe the excess earwax from your ear.

Re: I love LLMs, I hate hype

#184

Post by pojzon »

gyomu wrote: post #137
SwellJoe wrote: post #81

This line: "this is my main argument against the valuation of frontier labs. It’s not that AI won’t create that much value, it’s that they won’t capture it." That is a very astute and concise way to explain everything about how the frontier labs are behaving and how they're trying to push more people to pay token rates for the best models. At the current subscription prices ($100 or $200 a month for a generous, thoug…

In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.

Apple Watch with 1TB of vram with the size of well.. a watch.

Amazing story. If we make such leap in semiconductor field, it will be bigger than anything we have done till now. And all of that in 10years!

Re: I love LLMs, I hate hype

#185

Post by roncesvalles »

fragmede wrote: post #166
roncesvalles wrote: post #149

Earlier quoted context omitted.

>5.6 Sol, competitive with Fable (or close enough), and it's available via subscription (even the $20 subscription!) It's not comparable because OpenAI caps thinking to High in the ChatGPT "Chat" interface (and the "Work" thing where it actually does let us use Extra or Max is fucking shit). GPT 5.6 Sol (High) is almost certainly worse than Opus 4.8 (Extra), and nowhere close to Fable (Extra). I literally got a refun…

5.6 Sol isn't Fable, but calling it unusable is a bit much.

It's unusable only because I can't set 5.6 to Extra thinking in the "Chat" tab, and the "Work" feature that does let me use Extra is totally gimped for coding tasks. I'd rather use Opus 4.8 Extra/Max instead of Sol High (look up the benchmarks, thinking level is everything).

It is quite literally unusable to me.

Re: I love LLMs, I hate hype

#186

Post by throwup238 »

RajT88 wrote: post #136
cautiouscat wrote: post #19

Earlier quoted context omitted.

Those people are not just on Twitter. They’re here on HN, they’re at work, they’re at your next social gathering. I’ve found them to be unavoidable to some degree.

I see a lot of them on Nextdoor and at my city council meetings. Talking points like: "Data centers are just surveillance centers that are going to use AI to put us into a digital prison!" Whatever all that means. I assume some of it is about Flock cameras.

With their doors covered in Ring cameras.

Re: I love LLMs, I hate hype

#187

Post by [deleted] »

int_19h wrote: post #174
lacedeconstruct wrote: post #122

Earlier quoted context omitted.

>Makes perfect sense to anyone good at using these models. It doesn't really, because whenever I ask them what did they actually create, its always a shitty dashboard or a finance tracker or something derivative and worse than what is out there

I'd say that's more indicative of what kind of things people in your bubble generally work on. I vibe-coded a semantic parser for Lojban. A friend of mine is using it to work on dev tooling. Another friend, a mathematician, has recently used it to prove a conjecture he published 15 years ago.

[deleted]

Re: I love LLMs, I hate hype

#188

Post by kenforthewin »

int_19h wrote: post #178
kenforthewin wrote: post #92

Earlier quoted context omitted.

That's a much less generous interpretation of his writing. "Yes we will birth superintelligence, but everything will just sort of work out for us humans". This seems like a silly take to me.

Is it? To me, the notion that a superintelligence (by which I'm assuming you mean the more sensible "something smarter than us", not "literally a godlike entity") automatically means that sky is going to fall is sillier.

I have no idea what happens next if we create and horizontally scale superintelligence; one thing I do know for sure is it won't be "business as usual".

Re: I love LLMs, I hate hype

#189

Post by undeveloper »

LugosFergus wrote: post #151

Since no one is talking about it: T2 isn’t about machines taking over the world. That has happened (or will happen). But humans eventually defeat the machines. Skynet is trying to prevent that by killing John Connor. That’s what the movie is about. I suppose it’s also about John searching for a parental figure through the T800. He doesn’t get that through is foster parents and his estranged mother. Anyway, I don’t th…

he's biggest larper of all time

Re: I love LLMs, I hate hype

#190

Post by reinitctxoffset »

charcircuit wrote: post #139
reinitctxoffset wrote: post #90

Earlier quoted context omitted.

I'm pretty sure at this point that Anthropic is training mixture models (at least in the heavy pre-train) and deploying them dense with explicit loss on thinking trace coherence. Having a thinking trace that is legible, coherent, and immediately implies the explicit turn output and/or tool use seems difficult if not impossible to reliably get from mixture models. I predict MoE is a transitional technology, it's got t…

>I predict MoE is a transitional technology While scaling laws hold (more weights = better), and time / financial costs are not trivial the incentives are in place to have MoE. MoE means you can have more weights without increasing the critical path of evaluating it. I am curious what you believe the problems with it that would cause people to prefer using less weights. I'm not following what you mean by MoE can't ha…

Weights are not created equal: while interpretability is a young field the prevailing view at the moment is that MLP (hence experts) in a mixture model are substantially where dense encoding of factual information resides, attention is even less easily interpreted but it should be uncontroversial that temporal/sequential modeling occurs here.

So it's more consistent with available empirics to say that an architecture can be characterized along a spectrum from fully dense to mixture (a sub spectrum) to Engram-style lookup, and the amount of model power allocated at this point or that will recover different performance profiles.

By far the most stark example of how much performance in reasoning is left on the table is Qwen3.6-27B, which depending on the task, comparison model, and whose benchmarks you believe outperforms mixture models 15-60x larger in total parameter count.

It's badly under-studied (in public) because of the paucity of modern dense models at the near frontier, but even that one data point pretty much rules out the cocktail party version of the Chinchilla-adjacent scaling thesis (which wasn't about modern MoE to begin with).

The "Mixture of Parrots" work is a good jumping off point if you want to get a modern literature review.

Post reply on HN