Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fable and the end of the free lunch

dbreunig.com

Post reply on HN
Print view View original

71–80 of 268 posts

Re: Fable and the end of the free lunch

#71

Post by CamperBob2 »

ACCount37 wrote: post #67
bad_haircut72 wrote: post #58

Earlier quoted context omitted.

not an AI researcher - this is probably true for these "everything" LLMs but I think specialized models are gonna be the next big thing

"Specialized models" are a bit of a doozy. The biggest generalist models beat the most fine-tuned specialists, as a rule. You can bias an LLM away from literature knowledge and towards coding capabilities, but that buys you very little performance, and for too much effort. Generality and intelligence seem to be entangled very heavily in LLMs.

And yet, there's VibeThinker 3B to bring this long-held premise into question (if not to blast it to pieces.) It is practically illiterate by the standards of larger models, yet performs like models 100x its size on mathematical and logical reasoning tasks.

Re: Fable and the end of the free lunch

#72

Post by sipjca »

poincareball wrote: post #51
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

Evidence actually supports that capabilities are leveling off, and cheaper/faster is not really coming. Just log-linearly more capability at smaller parameter counts as they saturate.

what do you mean cheaper/faster is not really coming? the cost of the same level of intelligence steadily decreases year over year. computer hardware also advances at the same time enabling cheaper and faster serving (or move to local)

Re: Fable and the end of the free lunch

#73

Post by ForHackernews »

poincareball wrote: post #51
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

Evidence actually supports that capabilities are leveling off, and cheaper/faster is not really coming. Just log-linearly more capability at smaller parameter counts as they saturate.

Cheaper/faster is coming for sure.

Model on a custom silicon: https://chatjimmy.ai/

1-bit models that run on a CPU: https://github.com/microsoft/BitNet

Re: Fable and the end of the free lunch

#74

Post by ericol »

From my point of view the issue is that there are too many things wrong with Fable, making it seriously not worth the money.

For starters I don't know if it is an artifact of the model or something by design, but the level of gratuitous cognitive load carried by the complexity of its replies is unbearable.

Yes, it's a beast at coding, and also it's incredible nuanced at improving writing, validating specs, etc.

But when it comes to replying, it's the William Gibson of LLMs [1].

It has this tendency to take extreme detours to say things that could had been said in less, much simpler words. [2]

It really, really like to wrap very simple and atomic ideas on several layers of abstraction, building on unnecessary terms that carry no intrinsic information and assumes this vocabulary as shared and then building on top of it.

By the time I got to the end of the reply I'm bored to death and didn't understand even a third of what it told me.

I think the people at Anthropic should reflect on the maxim "You don't know a subject if you cannot explain it"

If you pardon my french, Fable is an insufferable obnoxious cunt.

---

[1] I apologize on the comparison but, as much as I love his first 2 trilogies, haven't been able to finish any of his last 2 books.

[2] "The residual you're accepting is the one from before: recovery currently rests on beneficial non-compliance, which may erode as models get more literal" == "We already accepted this risk"

" Its observable when it erodes is a stall that survives relaunch — loud at operator level, recoverable from the worklog, and fixable by codifying at that moment" == "When it breaks, it'll break visibly and recoverably"

"That is the iteration model applied exactly as written: resolve on first contact, don't pre-solve " == "So we fix it then, not now"

Re: Fable and the end of the free lunch

#75

Post by Tuna-Fish »

poincareball wrote: post #51
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

Evidence actually supports that capabilities are leveling off, and cheaper/faster is not really coming. Just log-linearly more capability at smaller parameter counts as they saturate.

Please explain why you think cheaper/faster is not coming?

All current devices used to run AI are very far from an efficient solution to the problem. What you really want is a pure dataflow architecture, instead of a von Neumann machine. The reason people aren't really making them yet is that when you build one, even if you use SRAM for the weights, you are binding yourself to the dimensions of the model you target -- your chip is only ever going to run variants of that specific model. And SRAM is much more expensive than ROM, so if you want to make a cheap version, you need to design a specific model into silicon.

Once model improvements taper off, the next thing that will happen is everyone will chase speed. There is no physical reason why a mid-sized model could not run at >1 million tokens per second on leading edge silicon, if all computation that can be parallelized, is. No-one will go straight to that, even for a mid-sized model that's like 20 distinct reticle-limited chips. But something like the next version of Taalas HC1 (presumably called HC2?) will probably boost a ~30B parameter model to ten of thousand of tokens+ per second from a single stream within 12 months.

Re: Fable and the end of the free lunch

#76

Post by glimshe »

geniium wrote: post #41
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

I was using ChatGPT voice during cooking to reflect on variations of a dishes i was preparing for years. It was so amazing to get advices and reflect that it struck me : I could use this model forever - it’s clever enough to help me tons and do lot of work for me - even if ai would stop evolving I would love it

All it needs is Internet access to remain useful with few shortcomings.

The next step would be automatic self-training. A free LLM that could access HN everyday (and the linked sites) for more data would remain current in programming for a really long time.

Re: Fable and the end of the free lunch

#77

Post by peteforde »

A few months ago folks were understandably annoyed when Microsoft dropped their heavily subsidized per-request pricing model because it was figuratively burning cash.

Well, I'm here to tell you that whatever is going on behind the scenes at Cursor with this Space-X acquisition in the works, the Auto setting is clearly routing all prompts through "Cursor Grok 4.6 High" right now.

This is a degree of subsidy that makes the Microsoft thing look quaint.

I reduced my $200/month subscription to the $20/month level and have proceeded to do what I would have paid about $1500 to do with Opus 4.7 or thereabouts, which is how Grok 4.6 High feels like it compares. I don't have anything remotely like hard evidence to back this estimate up beyond what I'm watching it do and I still somehow have ~10% of my monthly Auto capacity left on my account. It's completely nuts.

Can't say much more because I have more backlog to run before someone comes to their senses.

Re: Fable and the end of the free lunch

#78

Post by ACCount37 »

CamperBob2 wrote: post #71
ACCount37 wrote: post #67

Earlier quoted context omitted.

"Specialized models" are a bit of a doozy. The biggest generalist models beat the most fine-tuned specialists, as a rule. You can bias an LLM away from literature knowledge and towards coding capabilities, but that buys you very little performance, and for too much effort. Generality and intelligence seem to be entangled very heavily in LLMs.

And yet, there's VibeThinker 3B to bring this long-held premise into question (if not to blast it to pieces.) It is practically illiterate by the standards of larger models, yet performs like models 100x its size on mathematical and logical reasoning tasks.

Which are the kinds of tasks computers have been historically quite good at.

It's impressive that it does what it does, don't get me wrong. But if you expect it to replace the likes of GPT 5.6 Luna, let alone Sol? Nah.

Re: Fable and the end of the free lunch

#79

Post by robertjpayne »

peteforde wrote: post #77

A few months ago folks were understandably annoyed when Microsoft dropped their heavily subsidized per-request pricing model because it was figuratively burning cash. Well, I'm here to tell you that whatever is going on behind the scenes at Cursor with this Space-X acquisition in the works, the Auto setting is clearly routing all prompts through "Cursor Grok 4.6 High" right now. This is a degree of subsidy that makes…

Going to be great to see the cash burn on SpaceX's next earnings report. Will the cult keep the stock price pumped?

Re: Fable and the end of the free lunch

#80

Post by a2ff6eeb0 »

a2ff6eeb0 wrote: post #63
josephg wrote: post #61

Earlier quoted context omitted.

Yeah. Sometimes I wonder who the long term financial winners will be from the ai boom. It might be ram / gpu manufacturers. Or whoever cracks putting LLMs on asics.

It's going to be the shareholders of the first companies to crack AGI, and make human brains fully irrelevant economically. With the trillions of dollars that's going in through both investment and users, it's going to happen. I don't believe the human brain has fundamental magic that will make this impossible.

For the downvoters: What magic do you think the human brain has that makes it impossible to emulate acceptably?
Post reply on HN