31–40 of 142 posts
Fully open models really need to be a big part of the AI future. That includes all source code, open training data, how it's organized, fed to the model, processed, etc. Until that becomes a thing you're always going to be left wondering what exactly lies underneath the closed model you are using, leaving open the possibility for societal manipulation.
A bit off topic, but I think I'm starting to get model fatigue. These come out 10x faster than new Javascript frameworks were coming out 10 years ago (at least new models are far easier to adopt).
But over time, more and more people got into the chip-making business, and the big players started releasing more and more chips. Now only the die-hard CPU trackers worry about every new CPU and exactly how it's better ... while everyone else just worries about "which CPU will be good enough at this moment".
I think models are on that same arc.
It is great to see another player introduce a fully open stack. Nvidia's Nemotron is the only other prominent one I know of. All that said, the headline claims do not match the self-reported performance. For example, the dense 32B model is significantly behind Qwen3.8 27B (chart towards the bottom of https://ifm.ai/blog/k2 ). Gemma4 31B is not in the comparison set. This is the most important sweet spot for self host…
Earlier quoted context omitted.
The training data would need to have a permissive license for this to be possible.
Eventually we'll just construct 100% synthetic training data that can reliably reproduce pretrains and fine tunes. The first broadly useful fully open source models will do this. We already have open data / open code / open weights for some domain-specific cases, such as audio models trained on large open datasets, eg. Tacotron / LJSpeech from waaay back in the day, though that is certainly not SOTA anymore. Distilla…
Fully open models really need to be a big part of the AI future. That includes all source code, open training data, how it's organized, fed to the model, processed, etc. Until that becomes a thing you're always going to be left wondering what exactly lies underneath the closed model you are using, leaving open the possibility for societal manipulation.
Money is the issue here, no one wants to fund it.
Sure there are all kinds of problems with that situation. But it still demonstrates that they can be coerced: play nice or don't play at all.
Earlier quoted context omitted.
The training data would need to have a permissive license for this to be possible.
Eventually we'll just construct 100% synthetic training data that can reliably reproduce pretrains and fine tunes. The first broadly useful fully open source models will do this. We already have open data / open code / open weights for some domain-specific cases, such as audio models trained on large open datasets, eg. Tacotron / LJSpeech from waaay back in the day, though that is certainly not SOTA anymore. Distilla…
That said, I don't necessarily disagree with you. Talkie[1] presents an interesting case for it being at least possible to do this entirely on public domain material.
But even that used Claude somewhere in the course of its training pipeline (it's listed as a contributor on their GitHub), so again, how granular you want to get with that is still a question.
Fully open models really need to be a big part of the AI future. That includes all source code, open training data, how it's organized, fed to the model, processed, etc. Until that becomes a thing you're always going to be left wondering what exactly lies underneath the closed model you are using, leaving open the possibility for societal manipulation.
The training data would need to have a permissive license for this to be possible.
Decentralized unstoppable storage, combined with decentralized unstoppable training, sorta like SETI for AI training. The seed of this tech already exists with IPFS and others like it.
We know (some? all?) of the big labs have skirted copyright laws at one point or another. Truly open models would just build on what is publicly available.