Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fable and the end of the free lunch

dbreunig.com

Post reply on HN
Print view View original

211–220 of 268 posts

Re: Fable and the end of the free lunch

#211

Post by timmg »

chinathrow wrote: post #183
peteforde wrote: post #77

A few months ago folks were understandably annoyed when Microsoft dropped their heavily subsidized per-request pricing model because it was figuratively burning cash. Well, I'm here to tell you that whatever is going on behind the scenes at Cursor with this Space-X acquisition in the works, the Auto setting is clearly routing all prompts through "Cursor Grok 4.6 High" right now. This is a degree of subsidy that makes…

Giving Elon cash seems still wrong to me.

Worse than giving to Sama? (I mean that honestly.)

Re: Fable and the end of the free lunch

#212

Post by cosmojg »

bunderbunder wrote: post #199
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

But they don't really have that option. They're trapped in a Red Queen's race. The world keeps moving on, and so the models need to be retrained so that they can keep up with new information. Otherwise you'll get stuck with a model that only works well with information that existed prior to a dataset horizon that's receding into the past at a constant rate. At the same time, they have to keep iterating on the trainin…

Newer models[1] are being trained in ways that prioritize coding and agentic performance over raw knowledge[2] such that they increasingly rely on external tools for accessing hard data and information.

[1] https://artificialanalysis.ai/evaluations/omniscience?models...

[2] https://old.reddit.com/r/LocalLLaMA/comments/1vt7l3e/qwen382...

Re: Fable and the end of the free lunch

#213

Post by someothherguyy »

geniium wrote: post #41
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

I was using ChatGPT voice during cooking to reflect on variations of a dishes i was preparing for years. It was so amazing to get advices and reflect that it struck me : I could use this model forever - it’s clever enough to help me tons and do lot of work for me - even if ai would stop evolving I would love it

eventually the novelty wears off and depression kicks in

Re: Fable and the end of the free lunch

#214

Post by someothherguyy »

matteoraso wrote: post #38
nchmy wrote: post #2

The real revolution is Deepseek v4 flash and similar models (GPT 5.6 Luna, muse spark 1.2, mimo, etc...) - Genuinely good performance for a tiny fraction of the cost of Fable and even GLM etc... I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. Of course, both things continue to happen on a seemingly monthly basis

>I think a lot of people would be very content if they never got smarter, and just kept getting even cheaper/faster. There's a lot of truth to this. I think we're starting to approach the point where increased intelligence has declining marginal returns, such that it might not even be worthwhile to improve models unless it can be done cheaply.

i wish i was experiencing these things that everyone else is.

my experience is mostly frustration and rewrites of anything that requires more than what would take me an hour to do myself, unless it is pure translation / boiler plate work.

the leaps are there at getting to more "shaped" code (code that is correct for linters, static checking, etc), but i don't see the models exhibiting much intelligence. i really can't think of a time using LLMs for building anything where they did something that would make me go, "wow, that is really impressive, i wonder how it came up with that." just brute force search and pattern matching still.

even the interesting results in academic work seem to be more of a function of effort (proofs by exhaustion, fitting puzzle pieces in a search space, etc) than anything else. not to say people aren't using large language models to do impressive things, but the agents themselves do not seem very intelligent to me.

it feels like some engineering teams are aware of this fact and are driving agents using strict rule checks (like hooks on steroids), so they can drive some shape of output that aligns with what they require.

Re: Fable and the end of the free lunch

#215

Post by dboreham »

fidotron wrote: post #210
HighGoldstein wrote: post #200

I find it somewhat funny that the author starts by talking about how Moore's law enabled inefficient software and that we then had to make it more efficient, when almost all software today is horrendously inefficient compared to even 10 years ago, let alone 20-30. We've somehow even achieved a state where it doesn't matter how fast your CPU and memory are, the software will just perform horribly on any machine.

One of the funniest parts of the LLM wave is discovering that cron was so annoying to use that we will burn the planet to put an interface on it that people can actually work with. The tendency for absolute inefficiency is effectively unbounded until scarcity is imposed.

Assuming you are referring to using an LLM to generate crontab entries, is that a bad move? Seems like it removes the need for layers of UI that most of the time is never used. Same goes for regexes. Actually same for SQL. No need for layers to translate between what the user can specify and what's executed. Just type what you want to query for and the LLM generates the SQL.

Re: Fable and the end of the free lunch

#216

Post by dboreham »

penteract wrote: post #163
shawnb576 wrote: post #156

Earlier quoted context omitted.

Because LLMs don't understand anything. That's the tech. They can only predict what they have been trained with and fail daily at the most basic tasks. Granted they can do amazing things, no question there. But they are not "smart". For example, it seems that even at Fable scale, simple concepts like the passage of time or (gasp) timezones elude them. I live in UTC+10 and with any RFC8339 data LLMs are constantly con…

To me it sounds like you're repeating what gp said about the lack of online learning. Do you think that's insurmountble? Getting confused about timezones does not place LLMs behind that many humans. (But doing so repeatedly does highlight the lack of online learning).

I see this thread as progress because now there's three people saying this (seemed like it was just me for a year or two).

Re: Fable and the end of the free lunch

#217

Post by phplovesong »

HighGoldstein wrote: post #200

I find it somewhat funny that the author starts by talking about how Moore's law enabled inefficient software and that we then had to make it more efficient, when almost all software today is horrendously inefficient compared to even 10 years ago, let alone 20-30. We've somehow even achieved a state where it doesn't matter how fast your CPU and memory are, the software will just perform horribly on any machine.

Its slow because of human slop from the early to mi 2010s, and now slow because of AI was trained on the slop that existed.

Bottom line is the slop used to be manageable, but now there is 100x more code pushed, so that train has departed.

In the end its more bad code for features no one will use.

Re: Fable and the end of the free lunch

#218

Post by ninahaberl »

zdragnar wrote: post #209
ninahaberl wrote: post #208

Earlier quoted context omitted.

Models don't need to keep retraining just to stay current. Harnesses give them access to the internet, internal systems use RAGs, and so on. I think lower-cost models will get the largest piece of the pie, as with almost everything that has ever been sold. Just look at cars: US consumers buy the F-150, EU consumers buy the freaking Dacia Sandero the most :))) Ferrari/Lambo numbers are microscopic

Even with a harness, models don't reach out for new information they don't know about. For some tech, I have to have a local model draft a plan, then I have to adjust the plan to update it with the new API and references for where to find it. Even if I include that updated information in the prompt for the plan, the model says "what the user says is wrong, they probably meant this instead" and goes off in its own dir…

You are basically saying that some models (your local one, which is it?) in some setups (the API you mentioned) can fail to use fresh information if that conflicts with strong training priors. I agree:)

BUT

That's a bad model. My opinion is that for exactly this case we need to use RAGs/ APIs/ some retrieval mechanisms.

It's silly to train them on stuff that changes every week/month

I don't learn APIs by heart, I look them up. It's to expensive (my time) for me and (the compute) for the models

Re: Fable and the end of the free lunch

#219

Post by zdragnar »

ninahaberl wrote: post #218
zdragnar wrote: post #209

Earlier quoted context omitted.

Even with a harness, models don't reach out for new information they don't know about. For some tech, I have to have a local model draft a plan, then I have to adjust the plan to update it with the new API and references for where to find it. Even if I include that updated information in the prompt for the plan, the model says "what the user says is wrong, they probably meant this instead" and goes off in its own dir…

You are basically saying that some models (your local one, which is it?) in some setups (the API you mentioned) can fail to use fresh information if that conflicts with strong training priors. I agree:) BUT That's a bad model. My opinion is that for exactly this case we need to use RAGs/ APIs/ some retrieval mechanisms. It's silly to train them on stuff that changes every week/month I don't learn APIs by heart, I loo…

I most recently experienced this with Qwen 3.8 27b, though I've seen it on several other versions of their local models. It's also heavily biased towards digging into library source code rather than looking at API documentation.

To get it to the point of being remotely useful, I've had it start to write condensed fact blurbs into the agents.md file. It doubts itself so much and questions its every decision to the point that it'll literally blow the entire context on thinking alone in anything but the most basic CRUD projects otherwise.

What an earlier generation model would just start doing, it went out to research the source code in multiple libraries just to see if what it was thinking would work... then it said "Hey, I should really just do it" then went back and started researching more anyway, on and on (even on medium thinking level).

If there's a better local model for writing code, I'm all ears.

Re: Fable and the end of the free lunch

#220

Post by CamperBob2 »

ACCount37 wrote: post #129
CamperBob2 wrote: post #81

Earlier quoted context omitted.

Computers have historically been good at answering word problems fed to them verbatim?

No, but they were good at answering formalized versions of the same word problems. What this tells us is that a 3B LLM can retain enough NLU to understand those word problems. Which isn't particularly surprising? And also that the same LLM can solve a math or logic problem it understands. Which is a lot more impressive, because early LLMs were already quite good at NLU, but notoriously bad at things like math, logic…

"Formalizing the problem" is pretty much the whole shooting match. It takes intelligence to do that. The rest is mere calculation.
Post reply on HN