Interestingly, when opening this page, the first thought I had was not that the benchmarks should be high, but 'I really hope they did not benchmaxx'. I think a model with modest benchmark scores can have much better real world utility as opposed to the current frontiers that are RL'd into being robotic and rigid.
Inkling: Our Open-Weights Model
61–70 of 324 posts
Re: Inkling: Our Open-Weights Model
#62Earlier quoted context omitted.
> It could be but there are a host of companies going after open weights models: Arcee, Reflection, Llama (TBD on Meta's focus on closed-source versus open-source), etc. my bet is that Chinese government fund Chinese models way more compared to what those companies receive (except llama, which is outdated but was strong foundation at its time)
The story of Reflection AI is supposedly that the company was faffing and failing at winning in the coding agent space, but was introduced to Jenson, who suggested they build an open-weight model and said he would fund it. That turned into a $2 billion financing with NVIDIA doing roughly $500 million and was a complete pivot. I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of mone…
Jensen Huang is just trying to commoditize the complements to his GPUs.
Re: Inkling: Our Open-Weights Model
#63America needs its own DeepSeek or Z.ai, a lot of people (myself included) root for open chinese models to win because they have no other choice. Thinking Machines might be it.
What is the business model for an open weight model?
Re: Inkling: Our Open-Weights Model
#64America needs its own DeepSeek or Z.ai, a lot of people (myself included) root for open chinese models to win because they have no other choice. Thinking Machines might be it.
I don't hear about them a lot but it looks like arcee.ai is aiming to be just that.
Here are some of their current open weight offerings: https://www.arcee.ai/open-source-catalog
Re: Inkling: Our Open-Weights Model
#65Very nice, multi modal, largest open weight model that supports audio. Would be interesting to see how good the audio capability is.
If you want to run locally, checkout https://github.com/danielhanchen/llama.cpp/tree/add-inkling https://unsloth.ai/docs/models/inkling https://huggingface.co/unsloth/inkling-GGUF https://huggingface.co/unsloth/inkling-NVFP4
This supposedly is better than KimiK2.7, as much hype as GLM5.2 gets, I find myself using KimiK2.7 half of the time, so if the benchmark is true, then this can definitely go in the mix. My hope is that it might have strengths in some areas to beat all other open weight models.
Re: Inkling: Our Open-Weights Model
#66Raised 2 billion dollars at a 12 billion valuation and debuts at 41 on the Artificial Analysis Intelligence Index, while KIMI and DeepSeek will release Fable-class models this week. What a joke.
> ...while KIMI and DeepSeek will release Fable-class models this week. What new model is DeepSeek releasing? Their current V4 Pro at Max reasoning is consistently worse than GLM 5.2 at Max reasoning, though the latter is close to Opus 4.8 at Extra/Max reasoning, albeit a little bit worse in my experience (though if they gave comparable amounts of tokens to Anthropic 5x Max subscription I could see myself moving over…
DeepSeekV4 was a preview model, read the papers. It's not the final model. They released it to demonstrate architectural capabilities. They are still training and the model release is planned within the next month.
Re: Inkling: Our Open-Weights Model
#67Earlier quoted context omitted.
Just serving the model over API seems like a natural fit and is what many of them are doing. So simply being the cloud provider for your own open weight model can be a source of revenue
But so can everyone else. What’s the moat for spending all those billions. I understand the Chinese angle, they need to undermine American models as a matter of statecraft, but what is the business model here? It just seems like VC charity.
use open models to gain marketing/users/attention and then go closed? maybe
Re: Inkling: Our Open-Weights Model
#68Earlier quoted context omitted.
> If it's ~30% bigger and not as good as GLM 5.2, why would I tinker with this model? The benchmarks never tell the full story. Some of the open weights models have been benchmaxxed for a while. Their utility on real work can be different than the benchmark number. The multimodal input is also a big deal. Having vision input is really helpful for a lot of tasks.
Then why are they publishing the benchmarks which makes them look worse than GLM 5.2?
I'm not sure why I'm being downvoted but I didn't mean it in a negative way.
For such announcement, I would expect them to give me clues on when I should use this model and in which cases it's the best one.
The benchmarks that they share doesn't indicate that it's cheaper to run than other models, or can fit in my local machine, or excels in a specific vertical.
After reading the comments here and X, I can see it being the top-3 multi-modal open-source model though.
Re: Inkling: Our Open-Weights Model
#69America needs its own DeepSeek or Z.ai, a lot of people (myself included) root for open chinese models to win because they have no other choice. Thinking Machines might be it.
isn't that what Reflection is trying to be?
Re: Inkling: Our Open-Weights Model
#70It's nice to see a strong long context open weights model that is multi-modal. There are many applications that will benefit from the strength in audio here and until z.ai and co work in visual this could be very strong for general agentic applications, though I see there's a bit of weakness in the benches for areas that might make that less true. Like all models need to slap it in your harness and do proper evals on…
MiniMax M3 and DeepSeek v4-Pro are highly capable long context open weight multi-modal models. But long-context is a trap, because performance still falls dramatically after 150k-200k context.
> But long-context is a trap, because performance still falls dramatically after 150k-200k context.
I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.
This is why having good evals for the tasks you're working on is so important.
I do grant it's a good rule of thumb.