Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

github.com

Post reply on HN
Print view View original

1–9 of 9 posts

Re: Show HN: Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

#4

Post by tmzt »

This looks useful for somebody with a 16-32GB Mac Mini interested in running larger MoE models.

I've been working on a mesh environment that relies on an explicit prefix hash in the request and enables constructing a new session with a cached pre-filled system prompt specific KV-cache beyond what OpenAI-compatible APIs offer. Can you see a feature like that being supported?

Re: Show HN: Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

#6

Post by tmzt »

tmzt wrote: post #4

This looks useful for somebody with a 16-32GB Mac Mini interested in running larger MoE models. I've been working on a mesh environment that relies on an explicit prefix hash in the request and enables constructing a new session with a cached pre-filled system prompt specific KV-cache beyond what OpenAI-compatible APIs offer. Can you see a feature like that being supported?

I went ahead and forked this and added a new remote INFER protocol. You can see it at https://github.com/tmzt/ds4 if interested.
Post reply on HN