Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of open source AI

stateofopensource.ai

Post reply on HN
Print view View original

31–40 of 379 posts

Re: The state of open source AI

#31

Post by brunooliv »

foolswisdom wrote: post #26
brunooliv wrote: post #23

This is really insane to me. There's nothing practical about open-source models yet that makes them even remotely comparable to closed frontier models. All the hype around GLM, Qwen, now Kimi.... Are people really this naive that they believe these reports or, more worringly, are people NOT using these models and seeing the HUGE gap that still exists? Take a task, any medium-sized task, decently scoped that you'd tru…

> Take a task, any medium-sized task, decently scoped that you'd trust to give to Sonnet to finish without a hitch. Now give it to ANY open-source frontier model and watch them struggle and go in circles while failing tool calls and randomly assuming things. Claude used to be much worse than it is now, just as bad the open weights models are. And the open weights were worse. The labs will also try to keep the lead, b…

I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.

Re: The state of open source AI

#33

Post by mft_ »

babblingfish wrote: post #6

Speculation: open models is what will kill Anthropic and OpenAI. Hyperscalers can run the models without a licensing fee. Apple can make them smaller and put them on the device. The frontier models are an edge and a liability. They're astronomically expensive to train. Without them, their models will fade into obscurity. Their marketing depends on people believing the models are meaningfully different, as people have…

Open models are probably also comparatively astronomically expensive to train - just less so than the frontier models because they’re somewhat smaller, +/- the creators are more incentivised to focus on getting more from less compute because they’re have to, +/- they rely on distillation of the frontier models and this is more efficient.

But efficiencies aside; creation of open models still requires a lot of money and compute from a large organisation which is willing to accept zero return for that spend. This largesse is unlikely to continue forever; so the question is which will crack first, the frontier models’ business model or the fast followers’ generosity?

Re: The state of open source AI

#34

Post by urbsgpw »

100percentjake wrote: post #24
jdw64 wrote: post #7

The UI is really hard on the eyes. Personally, I think the font size is way too big, and the animation timing feels off. If this is a benchmark page and not a product page, I feel like the information should be scannable at a glance. The UX is bad.

I'm unsure what it is about AI developers seemingly not having eyeballs. The Hermes Agent website is absolutely eye-searing and the application itself resembles some sort of weird "RETVRN" greek-styled travel agent website. https://hermes-agent.nousresearch.com/

I use hermes only ever saw their repo. Atrocious. I was sure you were exaggerating.

Re: The state of open source AI

#35

Post by dosinga »

> Mozilla exists because one company tried to own the front door to the web, and an open community rose up to make sure it never could.

I'd say that the front door to the web is pretty much owned by Google and Apple at this point given Firefox current marketshare. And maybe that's enough, maybe a future where a low percentage of open models keep the rest of the system honest but that doesn't seem the argument of this article

Re: The state of open source AI

#36

Post by afavour »

mft_ wrote: post #33
babblingfish wrote: post #6

Speculation: open models is what will kill Anthropic and OpenAI. Hyperscalers can run the models without a licensing fee. Apple can make them smaller and put them on the device. The frontier models are an edge and a liability. They're astronomically expensive to train. Without them, their models will fade into obscurity. Their marketing depends on people believing the models are meaningfully different, as people have…

Open models are probably also comparatively astronomically expensive to train - just less so than the frontier models because they’re somewhat smaller, +/- the creators are more incentivised to focus on getting more from less compute because they’re have to, +/- they rely on distillation of the frontier models and this is more efficient. But efficiencies aside; creation of open models still requires a lot of money an…

I’m not exactly sure on the “how” but it only makes logical sense for (non-AI) companies to band together to fund the training of a shared model. Apple is a great example, AI is not their core business but they still require it.

The only thing that took us down a different path is the vast sums of VC funding pumped into the AI companies.

Re: The state of open source AI

#37

Post by bityard »

It sure is nice to see that Mozilla is still doing all that they can to keep on top of current trends, except developing a decent privacy-focused web browser for developers and power users.

Re: The state of open source AI

#38

Post by Jtarii »

100percentjake wrote: post #24
jdw64 wrote: post #7

The UI is really hard on the eyes. Personally, I think the font size is way too big, and the animation timing feels off. If this is a benchmark page and not a product page, I feel like the information should be scannable at a glance. The UX is bad.

I'm unsure what it is about AI developers seemingly not having eyeballs. The Hermes Agent website is absolutely eye-searing and the application itself resembles some sort of weird "RETVRN" greek-styled travel agent website. https://hermes-agent.nousresearch.com/

Really wish websites weren't allowed to force smooth scroll on. Hijacking basic browser functionality is so hostile.

Re: The state of open source AI

#39

Post by urbsgpw »

Haven't been following the articles and snippets we get from these labs about training their models for a while. But I'm guessing the latest chinese models are way less based on distilling? If not, then your speed of progress is still limited by the two labs (which we are collectively, in various forms subsidizing).

Re: The state of open source AI

#40

Post by fwip »

100percentjake wrote: post #24
jdw64 wrote: post #7

The UI is really hard on the eyes. Personally, I think the font size is way too big, and the animation timing feels off. If this is a benchmark page and not a product page, I feel like the information should be scannable at a glance. The UX is bad.

I'm unsure what it is about AI developers seemingly not having eyeballs. The Hermes Agent website is absolutely eye-searing and the application itself resembles some sort of weird "RETVRN" greek-styled travel agent website. https://hermes-agent.nousresearch.com/

Oh that's easy: they outsource design to the LLM, which doesn't have eyeballs.
Post reply on HN