Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

github.com

Post reply on HN
Print view View original

31–40 of 181 posts

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#31

Post by rwz »

0cf8612b2e1e wrote: post #20
logicallee wrote: post #15

Interesting project. The headline number (29 GB of RAM) is for 4k context. From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?

I saw someone’s excellent idea that if you have a slow system like this, you should communicate by email. It is no longer meant for realtime iteration, but more pointed questions for which there is more effort and time expected on both parties.

0.5t/s is still too slow even for email. For a moderately large inquiry (1MTok output, let's ignore the 4k context window limitation for now) it'll take the model around 23 days or uninterrupted execution to answer a single email.

Real world inquiries are gonna be much slower of course, but this setup is still too slow to do anything meaningfully useful I think.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#33

Post by sargunv »

gruez wrote: post #6
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

>Contributors >... >claude You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.

That's a strange and arbitrary line to draw. There's plenty of times I make edits by hand and then tell an agent to kick it up to a PR, or on the other extreme, let an agent implement something autonomously, and review the diff myself once it's in PR. Both of those scenarios involve the agent running the git commands, neither scenario indicates "the chances they've reviewed the code is slim"

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#35

Post by w45wsdfgdgdf »

cadamsdotcom wrote: post #32

Dear creator: you didn't ship the first draft of your code - why did you ship the first draft of your README??

because claude ships these verbose READMEs with its 'honest' takes and justifications for the naming. Its goal is to prime the next Agent that reads it, not you, human

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#36

Post by freedomben »

gruez wrote: post #6
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

>Contributors >... >claude You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.

I often let Claude write my commit messages even when I'm the one who wrote the code. Claude is often damn good at writing commit messages, and they frequently end up much better than if I wrote it all by hand. I nearly always edit them somewhat, but it's like starting from 80% instead of 0%. Some might call it laziness, but I call it working smarter rather than harder.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#37

Post by misterderpie »

marcobambini wrote: post #13
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

I wrote tons of software, even a programming language by hand https://github.com/marcobambini/gravity . I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.

> I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster.

The fact that the top comment on this thread calls it out, in a negative way, hints at that you aren't.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#38

Post by guybedo »

pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

i thought, here on HN, we were past the "oooh it's written by a LLM it's bad!".

I care about the craft, well designed systems, good clean architecture and code, etc...

But i also care about reaching goals. Whether i do it working on my own, or with human coworkers or with AI coworkers doesn't matter that much to me. Yes, the result is sometimes the most important thing.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#39

Post by outworlder »

misterderpie wrote: post #37
marcobambini wrote: post #13

Earlier quoted context omitted.

I wrote tons of software, even a programming language by hand https://github.com/marcobambini/gravity . I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.

> I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. The fact that the top comment on this thread calls it out, in a negative way, hints at that you aren't.

Non sequitur
Post reply on HN