Running Kimi K3 on a M1 Max

antirez wrote: post #4

SSD streaming on an M5 Max 128GB: https://x.com/antirez/status/2082136334160818528 Soon decent speed across two Mac Studios with 512GB of RAM.

wow cool! I like watching the new models come out and how they end up crammed in to run on local machines. I learned what mxfp4 is thanks to this latest Kimi release - although it sounds like it means that there's less room for compression in the model compared to others.

Re: Running Kimi K3 on a M1 Max

#7

Post by tito » Tue, Jul 28, 2026, 10:05 PM UTC

Azantys wrote: post #3

0.01 tk/s is unusable for anything, you would wait a whole day for just 1000 token of output, what is the point of projects like this?

I like seeing the latest and greatest model crammed into new systems to see how it fares. To deal with the speed, one person on reddit suggested using it in an email interface rather than a chat interface.

Re: Running Kimi K3 on a M1 Max

#8

Post by tito » Tue, Jul 28, 2026, 10:06 PM UTC

als0 wrote: post #5

Says it requires a 2TB disk? Must it be internal NVMe?

The Github specifically mentions an option to stream it from an online host. It's extremely slow.

Re: Running Kimi K3 on a M1 Max

#9

Post by nicechianti » Tue, Jul 28, 2026, 10:07 PM UTC

Azantys wrote: post #3

0.01 tk/s is unusable for anything, you would wait a whole day for just 1000 token of output, what is the point of projects like this?

[dead]

Re: Running Kimi K3 on a M1 Max

#10

Post by piterrro » Tue, Jul 28, 2026, 10:09 PM UTC

Will it fit on ESP32??