Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

github.com

Post reply on HN
Print view View original

21–30 of 181 posts

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#21

Post by Teever »

gruez wrote: post #6
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

>Contributors >... >claude You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.

That’s a needlessly antagonistic and insulting thing to say.

This person that you’ve never met and probably never will doesn’t owe any of us anything.

They’re out there doing what they want to do how they want to do it and if you don’t like it the correct response isn’t to insult them in front of a bunch of strangers on the internet for clout or whatever.

I doubt that you’d ever call them lazy to their face — why do it here?

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#24

Post by jpecar »

marcobambini wrote: post #12
jpecar wrote: post #10

Where can this 1tb k3.waste be downloaded?

It is not yet available, the only way is to download the official Kimi K3 model and then convert it: # 1. preflight: reachable? how big? does it fit? tools/fetch_weights.sh --dest /Volumes/staging/k3 --dry-run # 2. download — resumable, safe to kill, safe to re-run tools/fetch_weights.sh --dest /Volumes/staging/k3 # 3. convert into a container uv run --with torch --with safetensors python tools/convert.py \ --src /Vo…

Yeah, saw this ... was hoping that there's a torrent of it somewhere already. Or something.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#25

Post by chambored »

bglazer wrote: post #19
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant fo…

If that isn’t the perfect way to frame what I’ve seen and hated about LLM text, I don’t know what is. They certainly write for an audience with a historical context that almost no one has.

Re: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

#28

Post by andai »

bglazer wrote: post #19
pja wrote: post #4

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant fo…

>They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development

The transformer does not yet understand the non-transformer.[0]

This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.

I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.

[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.

Post reply on HN