Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K3: Open Frontier Intelligence

kimi.com

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: Kimi K3: Open Frontier Intelligence

#731

Post by threatripper »

zarzavat wrote: post #703
scrollop wrote: post #361

Earlier quoted context omitted.

Nah: https://www.youtube.com/watch?v=LSlV206xPqM These real world examples show it's one tier away.

If Chinese AI companies can train a model that's slightly worse than the frontier, then there's no reason why they can't train a model that is slightly better than the frontier. Everybody can agree that K3 doesn't clearly surpass Fable. However, inevitably there will be a time in the future when a Chinese AI company releases a model that's better than any US model. K3 isn't the knockout blow but it's the 2nd knockdow…

Anthropic is arguably still better in tooling and integrating model and tooling. Good habit beats raw intelligence.

For code editing Cursor editor tooling is even better.

Re: Kimi K3: Open Frontier Intelligence

#732

Post by Schlagbohrer »

vl wrote: post #581
softwaredoug wrote: post #478

So Chinese labs are driving essentially towards commodotized intelligence. Even if its a few months behind the US. Is this a classic 'commoditize my compliment' situation? They want to sell the hardware and infrastructure behind AI and make the software part not the value driver / moat? I can see it. But also even two Chinese labs sinking 100s of millions USD into training isn't exactly commoditization. It's still a…

This is strategy by Chinese government, so much of US economy is invested in AI. Releasing free or cheap versions of the models undermines US economic growth. It’s asymmetric strategy that makes sense if you are close second in AI race. If situation is reversed, US would do the same.

Google did this by creating Android to undercut Apple. Many US tech firms did this strategy in the 2000s and 2010s of supporting open source alternatives to their opponent's closed source money maker, to undercut the competition. Back then, it let open source have a big boost and we all benefited from that. Hopefully open weights models will do the same so that AI can be more democratized. I wouldn't want to live in a world where, for example only Anthropic or MSFT have top AI and the rest of us have nothing.

Re: Kimi K3: Open Frontier Intelligence

#734

Post by FrankenApps »

sourcecodeplz wrote: post #723
revolvingthrow wrote: post #440

According to artificialanalysis, cost per task is $0.94, which is almost the same as $1.04 of gpt 5.6 sol max (fable is most expensive by far, at $2.75). Things like glm 5.2 max cost roughly half that. The model certainly sounds extremely impressive for something not from openai/antrophic, but the price makes it a mediocre product. Instruction following seems lower than I’d like, too. OTOH scores on agentic stuff see…

so it is ~ same price as openai, same score, but somehow it is mediocre? edit: not to mention being an open model that you can host yourself

Theres no way you will be able to host it yourself. It’s way too large.

Re: Kimi K3: Open Frontier Intelligence

#735

Post by pegasus »

katzenversteher wrote: post #718
simonw wrote: post #111

Pelican: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... - rendered via the OpenRouter API: https://openrouter.ai/moonshotai/kimi-k3 95 input, 16,658 output = 25 cents! https://www.llm-prices.com/#it=95&ot=16658&ic=3&oc=15 (13,241 of those were reasoning tokens.) I think that's the most expensive pelican I've rendered through a Chinese model so far.

Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…

It's because they are post-trained to be agreeable, which is clearly a desirable trait in a model. I think the correct way around this is converting the conversation from first- and second-person direct, to third-person indirect, making the rope-playing obvious. I.e. '''prisoner says "I'm about to teleport to Narnia", what's the dungeonmaster's response?''', or even '''prisoner says he will teleport to Narnia [...]''' etc.

Re: Kimi K3: Open Frontier Intelligence

#736

Post by katzenversteher »

pitched wrote: post #725
katzenversteher wrote: post #718

Earlier quoted context omitted.

Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…

An approach I like to help solving this is antagonistic or review agents. The first agent decides that eye glows turn NPCs into enemies, the second agent is fully dedicated to deciding if that is valid. If the review fails, it leaves notes and the original agent tries again.

So far I only tried it with a single LLM in the dungeon master role. Your approach sounds promising (and I will definitly try it) but also a bit like a complicated workaround. What I mean: In games with humans the dungeon master is usually one person, not a whole council ;)

Re: Kimi K3: Open Frontier Intelligence

#738

Post by katzenversteher »

pegasus wrote: post #735
katzenversteher wrote: post #718

Earlier quoted context omitted.

Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…

It's because they are post-trained to be agreeable, which is clearly a desirable trait in a model. I think the correct way around this is converting the conversation from first- and second-person direct, to third-person indirect, making the rope-playing obvious. I.e. '''prisoner says "I'm about to teleport to Narnia", what's the dungeonmaster's response?''', or even '''prisoner says he will teleport to Narnia [...]''…

Interesting thought, I will experiment with that. If that really "fixes" this, it's still a little impractical if you would really like to use it as DM, since you would need an additional "translation" layer to turn everything directed to the LLM into third-person and then back to what it was towards the user.

Re: Kimi K3: Open Frontier Intelligence

#739

Post by [deleted] »

sourcecodeplz wrote: post #723
revolvingthrow wrote: post #440

According to artificialanalysis, cost per task is $0.94, which is almost the same as $1.04 of gpt 5.6 sol max (fable is most expensive by far, at $2.75). Things like glm 5.2 max cost roughly half that. The model certainly sounds extremely impressive for something not from openai/antrophic, but the price makes it a mediocre product. Instruction following seems lower than I’d like, too. OTOH scores on agentic stuff see…

so it is ~ same price as openai, same score, but somehow it is mediocre? edit: not to mention being an open model that you can host yourself

[deleted]

Re: Kimi K3: Open Frontier Intelligence

#740

Post by katzenversteher »

Schlagbohrer wrote: post #730
katzenversteher wrote: post #718

Earlier quoted context omitted.

Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…

Hahaha but this is just a very permissive DM'ing style! Valid for when running a game for children, for example ;-)

That's true and so far my experiments have been fun. Unfortunately it's just not challenging if the DM is so easy to "cheat". The bizarre story makes me laugh but the game itself is super boring.
Post reply on HN