Replace America with „The World“
Open source AI must win
451–460 of 538 posts
Re: Open source AI must win
#452Earlier quoted context omitted.
https://tinygrad.org/#tinybox I'm not sure exactly why you would buy through them vs rolling your own if you could afford the equivalent hardware. I'm a firm supporter of local inference though so good on them for doing something
Lol I get nervous when I see a list of products with full specs but no prices
They have prices. Click on the links in the shipping row.
Re: Open source AI must win
#453Re: Open source AI must win
#454Earlier quoted context omitted.
> it's likely they're distilled from the same source Any credible references for this? The implication that Anthropic has an even bigger and better model that they haven't released is hard to believe.
Lab folks keep cards close to their chests here, but it's likely Mythos was an earlier teacher model for Opus that got additional cybersec post-training. Whether they have a bigger tier than that is hard to say, labs have been cautiously scaling parameters since the failure of GPT4.1. They 100% have a bigger/better model they haven't released, but that's probably more down to it not being done cooking yet. Once it's…
Appreciate the long answer. Why is it more likely that Gemini 3 Pro/Flash/Lite are distillations of the same parent model than that they’re different training runs on the same dataset, with minor version bumps being different post-training setups?
Re: Open source AI must win
#455Earlier quoted context omitted.
These things sound plausible, but have they actually been demonstrated? Wouldn't anyone who succeeded in making such a small but useful LLM be raking in the money now?
Cursor's composer 2.5 is a perfect example. It's right on the heels of the frontier (for coding only) for an order of magnitude cheaper. As much as I've shit on Cursor in the past, I do think the company is well positioned to pick up people getting sticker shock on Anthropic tokens, if they can get their marketing down.
If that's Kimi-based it would very much be on the larger side of open-weight models (1T params).
Re: Open source AI must win
#456Earlier quoted context omitted.
Ya that'd be an awesome project, the only issue is how do you verify it's not being poisoned? To actually validate it would require more analysis than the training took to run. It would require a trusted network, not an open one, unless that can get solved somehow.
Make multiple nodes do the same job, compare results.
[deleted]
Re: Open source AI must win
#457Earlier quoted context omitted.
DeepSeek and GLM (plus Kimi) are at or above Sonnet level wrt. favorable workloads like coding. They're not close to Opus or the latest GPT yet, and Fable is even higher than that. Other workloads relying more on real-world knowledge have them even further behind, and this can't be mitigated without making the model itself bigger and harder to host locally.
Not true. Big models buy you baked in knowledge and long context cohesion. A model can be trained to use search and knowledge base tools more efficiently to mitigate the former, and harnesses/workflows can be designed to push models into small parallel threads to mitigate the latter. The thing that big models will always bring to the table is the ability to YOLO weak/under-specified prompts, and spend less time in th…
Knowledge-base access is not very useful in general because a model doesn't have well-defined "known unknowns" that might trigger an agentic search of the outside knowledge base. Plus surfacing knowledge you don't know much about is itself hard.
Re: Open source AI must win
#458I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable. And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datu…
there are some strong open source groups like NOUS research taking the fight https://nousresearch.com/
It seems this project is serious and very promising. They have the Psyche network which seems real and operational. They're able to produce ~50B-class models, this will only grow over time of course. Very cool.
Re: Open source AI must win
#459This is really a feel good argument and I agree with what he’s saying in principle but it offers zero in terms of a practical strategy or stable state where this is feasible. If you want to jump on the bandwagon then let’s put our pants on and offer a concrete suggestion that is practical and coherent. Otherwise what are we doing. Does anyone have suggestions to that effect?
What’s the world in which frontier model performance is open source? What does that look like? What’s a sensible business model that makes this sustainable? What’s a sensible regulatory framework that doesn’t hamstring AI progress?
Everyone is so enamored with these Chinese lab models like deepseek and qwen and GLM but they exist in a world where the top performance is still claimed by closed source models. These are not developed out of any benevolent commitment to the principles laid out in this article. A world in which OSS is the frontier and its development is controlled and funded by government subsidies of an autocratic government is not reassuring. You can inspect weights but good luck getting the cat back in the bag in terms of capabilities, safeguards, value system, bias, nerfing if it smells American business use cases.
Deepseek was such a darling but guess what, it’s now raising money — 300M at 10 billion valuation. OSS development isn’t sustainable as a business model and in a world where it costs a few hundred million to develop a frontier model, you need a strong business model, or you need strong state subsidies and incentives which introduce a billion new problems.
the most sensible economic picture of OSS models already exist. Commoditize your complement, passion projects for a hedge fund. These are unsustainable and exist at the pleasure of the business or the founder.
Re: Open source AI must win
#460…just crowdsource your own data centre…