271–280 of 538 posts
So I've long said that the valuation of OpenAI at a trillion(ish) dollars depends on OpenAI "winning" and "owning" AI and there being a sufficient moat to stay ahead of competition. Without that, the company is worth a fraction of that. Anthropic is probably positioned better here actually but it's still kinda true there too. Ever since a Chinese firm released DeepSeek I immediately came to the realization that any U…
Good Guy China! :DDD
I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…
I always wondered if 1000 1M parameter models fine-tuned to specific tasks with a small router could perform as well as 100B models.
And I know this is roughly how MoE works, but current MoE models still require training the model as a whole, and big players don’t have an incentive to change that.
But OpenSource community does…
Earlier quoted context omitted.
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea. The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consu…
What makes you think Deepseek or GLM won't catch up to Fable level? Why would there be a break in the trend now?
One being that extrapolating from like 3 data points is hardly science. All trends break at some point.
The other is that the measures to prevent distillation of their models (if it was a secret sauce of Chinese models) could work if nobody is allowed to use them.
Earlier quoted context omitted.
Nvidia and "open source" is like opposite things. Nvidia only ever opened stuff that helps their bottom line or improve vendor lock-in. But yeah they are good shovel seller and competitor to actually evil companies that literally wants to eat all the world chips and energy supply.
In the open source space, the Nemotron models from nVidia are quite real. Including a Nemotron Ultra variety meant to be large enough for near-SOTA.
Basically everything Nvidia does in open source is there to make sure their proprietary stack have a good moat and no competitor stack can catch up.
One day an open source model reaches "good enough" level. Maybe around the level the current frontier has and most people will use that
Earlier quoted context omitted.
Nvidia and "open source" is like opposite things. Nvidia only ever opened stuff that helps their bottom line or improve vendor lock-in. But yeah they are good shovel seller and competitor to actually evil companies that literally wants to eat all the world chips and energy supply.
Strongly disagree: https://build.nvidia.com/models Their license terms are also incredibly generous and allow commercial use, modification, etc, at no cost.
We're saved /s
Instead of doing a vanity site with a shelf-life of a few days, see where the action already is in online local LLM research and communities and contribute.
/s