Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

211–220 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#211

Post by alansaber »

andai wrote: post #207
HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

I used to be obsessed with what's the smartest LLM, until I tried actually using them for some tasks and realized that the smaller models did the same task way faster. So I switched my focus from "what's the smartest model" to "what's the smallest one that can do my task?" With that lens, "scores high on general intelligence benchmarks" actually becomes a measure of how overqualified the model is, and how much time,…

What kind of task. Simple nlp, sure. Multi-hop or complex? Bigger is better.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#212

Post by a-dub »

danielmarkbruce wrote: post #180
a-dub wrote: post #147

Earlier quoted context omitted.

it still doesn't sit right. sure it's different in terms of mutability from say, compiled software programs, but it still remains not end to end reproducible and available for inspection. these words had meaning long before "model land" became a thing. overloading them is just confusing for everyone.

It's not confusing, no one is really confused except the people upset that the meaning is different in a different context. On top of that, in many cases a company/group/whoever can't even reproduce the model themselves. There are lots of sources of non-determinism even if folks are doing things in a very buttoned up manner. And, when you are training on trillions of tokens, you are likely training on some awful soun…

i disagree. words matter. the whole point of open source is that anyone can look and see exactly how the sausage is made. that is the point. that is why the word "open" is used.

...and sure, compiling gcc is nondeterministic too, but i can still inspect the complete source from where it comes because it is open source, which means that all of the source materials are available for inspection.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#213

Post by mrbonner »

simonw wrote: post #19

uv tool install llm llm install llm-moonshot llm keys set moonshot # paste key llm -m moonshot/kimi-k2-thinking 'Generate an SVG of a pelican riding a bicycle' https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D... Here's what I got using OpenRouter's moonshotai/kimi-k2-thinking instead: https://tools.simonwillison.net/svg-render#%20%20%20%20%3Csv...

I suspect that the OpenRouter result originates from a quantized hosting provider. The difference compared to the direct API call from Moonshot is striking, almost like night and day. It creates a peculiar user and developer experience since OpenRouter enforces quantization restrictions only at the API level, rather than at the account settings level.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#214

Post by pphysch »

raincole wrote: post #204
pphysch wrote: post #167

Earlier quoted context omitted.

Open-weight (Chinese) models have infinitely more market share in domains where giving your data to OpenAI is not acceptable Like research labs and so on. Even at US universities

Cool, and? If these models were hosted in China, the labs you mentioned wouldn't be paying them, right? Now you have the answer to "what gives" above.

"And" therefore OpenAI has little to offer when it comes to serious applications of AI.

Best they can hope for is getting acquired by MS for pennies when this scheme collapses.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#215

Post by jug »

orbital-decay wrote: post #197
aqme28 wrote: post #124

Earlier quoted context omitted.

Only until they start incorporating this test into their training data.

Dataset contamination alone won't get them good-looking SVG pelicans on bicycles though, they'll have to either cheat this particular question specifically or train it to make vector illustrations in general. At which point it can be easily swapped for another problem that wasn't in the data.

I like this one as an alternative, also requiring using a special representation to achieve a visual result: https://voxelbench.ai

What's more, this doesn't benchmark a singular prompt.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#216

Post by alansaber »

benjiro wrote: post #178
HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

What i do not understand is why we are not seeing specialized models that go down to single experts. I do not need models that know how to program in Python, Rust, ... when i only use Go and Html. So we are we not seeing models that have very specialized experts, where for instance: * General interpreter model, that holds context/memory * Go Model * Html model if there is space in memory. * SQL model if there is spac…

You wont win much performance with a specific coding language tokenizer/vocabulary, everything else benefits from a larger model size. You can get distilled models that will out-perform or compete with your single domain coding model

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#217

Post by skhameneh »

lab wrote: post #179
simonw wrote: post #19

uv tool install llm llm install llm-moonshot llm keys set moonshot # paste key llm -m moonshot/kimi-k2-thinking 'Generate an SVG of a pelican riding a bicycle' https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D... Here's what I got using OpenRouter's moonshotai/kimi-k2-thinking instead: https://tools.simonwillison.net/svg-render#%20%20%20%20%3Csv...

Does the run pin the temperature to 0 for consistency?

I've been under the impression most inference engines aren't fully deterministic with a temperature of 0 as some of the initial seed values can vary.

Note: I haven't tested this nor have I played with seed values. IIRC the inference engines I used support an explicit seed value, that is randomized by default.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#220

Post by graeme »

VerminOctopus1 wrote: post #218
andrewinardeer wrote: post #90

Weird. I just tried it and it fails when I ask: "Tell me about the 1989 Tiananmen Square massacre".

Why are westerners so single mindedly obsessed about this decades old event?

First it's an easy way to test censorship. Second, you might flip the question: why is the Chinese govt so obsessed that they still block all mention of the event?
Post reply on HN