11–20 of 63 posts
LLMs are only going to improve by fragmenting them into specialized systems for low parameter high performance results. We’ve reached the point where models will get smaller and more compact
So its a model designed exclusively for a purpose? Then the results should not be that surprising. It's still impressive don't get me wrong.
The results are interesting for showing the efficacy of small, fine-tuned models that can be run locally. AI providers as a business need their do-all models to be better than these if they want long-term revenue through the APIs, right?
https://github.com/herniqeu/extract0
To quote Mulder: I want to believe.
LLMs are only going to improve by fragmenting them into specialized systems for low parameter high performance results. We’ve reached the point where models will get smaller and more compact
Open-Source style small players will actually solve problems with AI.
And the big money invested things are going to do stupid pointless bubbly things at best, or enshittify other good things at worst.
Govern yourselves accordingly.
I guess this is a small step forward, if nothing else, to the day when I can actually teach a model something in situ on my personal machine (notice I said machine, not "machines") in a very short amount of time. I feel that until then, LLMs and similar technologies won't be maximally helpful. They're very useful, but not maximally helpful.
There is so much research that shows you can beat frontier models with very little investment. It's confusing that the industry at large hasn't caught up with that
It really seems like all the next big leaps in AI are going to be fine-tuning fit-for-purpose models. Everything past GPT5 has been ... fine. It's better at chat (sort of, depending on your tone preferenc) and way better at coding/tool use. In our product (plan out a migration with AI), they've gotten worse, because they want to chat or code. I'd have expected the coding knowledge to generalize, but no! Especially Cl…
LLMs are only going to improve by fragmenting them into specialized systems for low parameter high performance results. We’ve reached the point where models will get smaller and more compact
I thought "The Bitter Lesson" was that whole a specialised system will outperform in the short term, generalized systems with lots of data win in the long term. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
But it's informative for the engineers that need something right now, because it means taking the best general purpose tool and specializing it will outperform the general tool, and you can sustain that if you are willing to always hop tools and respecialize. As we may.