Live data from Hacker News

Kimi K2.5 Technical Report [pdf]

github.com

51–60 of 146 posts

Re: Kimi K2.5 Technical Report [pdf]

#51
I'm not very familiar with this model series but tried it when prototyping a potential 'Vibesbench' (my eval) conversation I tried it. Here's Kimi on a Robocop scene

kimi k2 thinking turbo

"He’s not being poetic; he’s weaponizing nostalgia. Quoting Casablanca—the ultimate “beautiful sacrifice” romance—to taunt Morton about corporate backstabbing is pure executive sleaze. It’s the same move as his mahogany desk and flag pin: draping sentimental Americana over naked power. He’s essentially saying, “Isn’t it romantic, Bob? You fought the good fight, now you die.” "

grok 4.1

"So when Dick Jones smirks and says, “Time goes by, it’s the same old story… the fight for love and glory, huh, Bob?” that’s pure Ed Neumeier flexing his love of old Hollywood while simultaneously mocking how these psychopathic ’80s execs think they’re in some grand cinematic drama instead of just murdering a guy for market share."

chatgpt-4o

"The reason the Casablanca reference matters is RoboCop itself is partially a noir—and also a tragedy built inside a genre engine. [… ] So yeah, calling back to Casablanca is not just some meta-joke—it’s a knowing wink that RoboCop is a noir trapped in cybernetic skin. // Even Clarence Boddicker, for all his cartoonish sadism, plays like a twisted stand-in for a noir heavy—talking smooth, dropping nicknames, throwing out casual violence like it’s a punchline."

opus 4.5

"Jones is performing avuncular menace. He's the kind of executive who'd quote old movies while having you murdered, because it's all performance to him, all style. The substance is just power."

Re: Kimi K2.5 Technical Report [pdf]

#52
post #48
post #14

Earlier quoted context omitted.

I'm not running it locally (it's gigantic!) I'm using the API at https://platform.moonshot.ai

How long until this can be run on consumer grade hardware or a domestic electricity supply I wonder. Anyone have a projection?

You need 600gb of VRAM + MEMORY (+ DISK) to fit the model (full) or 240 for the 1b quantized model. Of course this will be slow.

Through moonshot api it is pretty fast (much much much faster than Gemini 3 pro and Claude sonnet, probably faster than Gemini flash), though. To get similar experience they say at least 4xH200.

If you don't mind running it super slow, you still need around 600gb of VRAM + fast RAM.

It's already possible to run 4xH200 in a domestic environment (it would be instantaneous for most tasks, unbelievable speed). It's just very very expensive and probably challenging for most users, manageable/easy for the average hacker news crowd.

Expensive AND hard to source high end GPUs, if you manage to source for the old prices around 200 thousand dollars to get maximum speed I guess, you could probably run decently on a bunch of high end machines, for let's say, 40k (slow).

Re: Kimi K2.5 Technical Report [pdf]

#53
post #40
post #18

Earlier quoted context omitted.

Just curious - how does it compare to GLM 4.7? Ever since they gave the $28/year deal, I've been using it for personal projects and am very happy with it (via opencode). https://z.ai/subscribe

Is the Lite plan enough for your projects?

Very much so. I'm using it for small personal stuff on my home PC. Nothing grand. Not having to worry about token usage has been great (previously was paying per API use).

I haven't stress tested it with anything large. Both at work and home, I don't give much free rein to the AI (e.g. I examine and approve all code changes).

Lite plan doesn't have vision, so you cannot copy/paste an image there. But I can always switch models when I need to.

Re: Kimi K2.5 Technical Report [pdf]

#54
post #19

I wonder how K2.5 + OpenCode compares to Opus with CC. If it is close I would let go of my subscription, as probably a lot of people.

I also wonder if CC can be used with k2.5 with the appropriate API adapter

yes, just use the base url https://api.moonshot.ai/anthropic

(https://platform.moonshot.ai/docs/guide/agent-support#config...)

Re: Kimi K2.5 Technical Report [pdf]

#55
post #48
post #14

Earlier quoted context omitted.

I'm not running it locally (it's gigantic!) I'm using the API at https://platform.moonshot.ai

How long until this can be run on consumer grade hardware or a domestic electricity supply I wonder. Anyone have a projection?

You can run it on consumer grade hardware right now, but it will be rather slow. NVMe SSDs these days have a read speed of 7 GB/s (EDIT: or even faster than that! Thank you @hedgehog for the update), so it will give you one token roughly every three seconds while crunching through the 32 billion active parameters, which are natively quantized to 4 bit each. If you want to run it faster, you have to spend more money.

Some people in the localllama subreddit have built systems which run large models at more decent speeds: https://www.reddit.com/r/LocalLLaMA/

Re: Kimi K2.5 Technical Report [pdf]

#57
post #18
post #14

Earlier quoted context omitted.

I'm not running it locally (it's gigantic!) I'm using the API at https://platform.moonshot.ai

Just curious - how does it compare to GLM 4.7? Ever since they gave the $28/year deal, I've been using it for personal projects and am very happy with it (via opencode). https://z.ai/subscribe

The old Kimi K2 is better than GLM4.7

Re: Kimi K2.5 Technical Report [pdf]

#58
post #48
post #14

Earlier quoted context omitted.

I'm not running it locally (it's gigantic!) I'm using the API at https://platform.moonshot.ai

How long until this can be run on consumer grade hardware or a domestic electricity supply I wonder. Anyone have a projection?

You can run it on a mac studio with 512gb ram, that's the easiest way. I run it at home on a multi rig GPU with partial offload to ram.

Re: Kimi K2.5 Technical Report [pdf]

#59
Sorry if this is an easy-answerable question - but by open we can download this and use totally offline if now or in the future if we have hardware capable? Seems like a great thing to archive if the world falls apart (said half-jokingly)

Re: Kimi K2.5 Technical Report [pdf]

#60
post #30

Earlier quoted context omitted.

To save everyone a click > The 1.8-bit (UD-TQ1_0) quant will run on a single 24GB GPU if you offload all MoE layers to system RAM (or a fast SSD). With ~256GB RAM, expect ~10 tokens/s. The full Kimi K2.5 model is 630GB and typically requires at least 4× H200 GPUs. If the model fits, you will get >40 tokens/s when using a B200. To run the model in near full precision, you can use the 4-bit or 5-bit quants. You can use…

I'm running the Q4_K_M quant on a xeon with 7x A4000s and I'm getting about 8 tok/s with small context (16k). I need to do more tuning, I think I can get more out of it, but it's never gonna be fast on this suboptimal machine.

you can add 1 more GPU so you can take advantage of tensor parallel. I get the same speed with 5 3090's with most of the model on 2400mhz ddr4 ram, 8.5tk almost constant. I don't really do agents but chat, and it holds up to 64k.
Post reply on HN