61–70 of 172 posts
LLMs is quickly became commodities. US AI labs like OpenAI is losing their moat. Hyperscalers and data center owners who can sell cheap token will win
A bit of topic. But how likely is it that the US will restrict Chinese open weight models and also force Euro countries to do the same? I think it will be effective within 6 months. The US is having a hard time staying competitive.
It isn't legally possible for them to do this at the EU level. The EU parliament would never vote for it. For pressure at the country level leading to this kind of thing I think it's very unlikely. Here in Sweden it wouldn't just require a vote in the Swedish parliament and before this there'd have to be förarbeten and you can't just brazenly push things through with insane arguments, Swedish social convention goes a…
[flagged]
Earlier quoted context omitted.
Since the model is open-weights, you can get from other providers, for example see https://openrouter.ai/moonshotai/kimi-k3#providers
The only downside with third party providers is that you have to trust that the provider have setup and configured it correctly, and is not secretly quantizing it. See Kimi Vendor Verifier.
Earlier quoted context omitted.
I'm always curious what you guys are working on; every git repo I've run a local model on and stick below <100k to increase speed seems effective enough to scope patches and changes.
Try doing a refactoring of some sort or larger new feature using just an agent on a moderately sized codebase, 256k will be compacting every few minutes, and result will be unusable.
Wow. So kimi is suddenly half the price for all users until they hit 256k of context? Thats massive.
Edit: I was wrong, thanks to longwave for pointing this out. It's absolutely possible to start out on the 256k model and then switch to the 1 million model when you get close to the context limit without invalidating the cache:
"When switching from k3-256k to k3 (1M), if k3-256k is close to the 256k limit and you don't want compact to lose information, you can switch directly to 1M. The current version switching from 256k to 1M does not affect the cache."
Wow. So kimi is suddenly half the price for all users until they hit 256k of context? Thats massive.
Earlier quoted context omitted.
My current Claude Code session has been going on for like 35 hours and has used up around 400 million tokens, thankfully almost all of those being cached (95-98%) - pretty typical for long form agentic work. First you spend like 2-3 hours working on a plan, once you have that you just tell the model to go and implement it, do adversarial sub-agent review loops before each commit and also make sure that all tooling an…
Once you have the plan you don't need to keep the 2hrs of research in the context (which is most of it) you can drop that plan into a file and start fresh for implementation.
If I feel like the model and I explored a lot of options I won't want to keep context as it might be confusing.
I think the more you use it the better judge you are of whether you should purge, compress, or just keep the context before executing the plan.
LLMs is quickly became commodities. US AI labs like OpenAI is losing their moat. Hyperscalers and data center owners who can sell cheap token will win
I believe Codex harness is pretty sticky though I haven’t tried many others. Does anyone else provide a harness of that quality?