none of these local models are good for development, complete waste of time. nobody has $100k+ hardware sitting around at home to actually run a good model
Qwen 3.6 27B is the sweet spot for local development
21–30 of 809 posts
Re: Qwen 3.6 27B is the sweet spot for local development
#22Strix Halo user here. While Qwen 3.6 27B exhibits remarkable intelligence density, I will still take unsloth's dynamic IQ2_XXS of Minimax M2.7 over Q8_0 Qwen 3.6 27B any day of the week, and this isn't just because of generation speed either. I wrote my own custom harness, and I get hallucinated tool call parameters and bizarre invocations with Q3.6 27B even at Q8_0, but no issues with the IQ2_XXS of M2.7.
Re: Qwen 3.6 27B is the sweet spot for local development
#23> ... on my Macbook Max M5 128 GB Local development for who ? How many of y'all are rocking 128GB of memory? Am I reading Apple's site correctly that it's a $10,000 laptop?
It wasn't $10k a month ago
Re: Qwen 3.6 27B is the sweet spot for local development
#24none of these local models are good for development, complete waste of time. nobody has $100k+ hardware sitting around at home to actually run a good model
skill issue
Re: Qwen 3.6 27B is the sweet spot for local development
#25Call me back when you can run these models on 16GB of RAM and any recent i5/i7. Until then, there’s no point on using these toy models.
Re: Qwen 3.6 27B is the sweet spot for local development
#26The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0]
Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs.
[0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...
Re: Qwen 3.6 27B is the sweet spot for local development
#27This is kind of like saying grass is green to be honest
Like everybody got 128 GB RAM..
I've been running it almost since launch on a 3090 (24gb vram), you really don't need that much. Second hand those are really cheap and i get 50-70 t/s (with MTP at 2), full ctx. IQ4_NL (unsloth) on this model seems suspiciously competent, and after the (by now not so recent) updates to q4 KV on llama.cpp, I just keep going back to it after dsv4pro disappointed me for the 100th time because it gave up on a task.
Re: Qwen 3.6 27B is the sweet spot for local development
#28Call me back when you can run these models on 16GB of RAM and any recent i5/i7. Until then, there’s no point on using these toy models.
You need it to run in about 8 GB so you have extra space for the context window.
Re: Qwen 3.6 27B is the sweet spot for local development
#29Spent a week trying to get sensible results out of llama 3.3 At one point it even simulated doing the work, log output and everything and when I challenged it about the missing artefacts it actually started questioning my intelligence. Seems appropriate for a Zuck enterprise. Qwen on the other hand got straight to work with astonishing competency on the same system. From what I read llama3 needs beefier compute to re…
You might find this helpful. llama is not anywhere near the Pareto distribution (performance vs cost) https://arena.ai/leaderboard/code/webdev/pareto?license=open... https://arena.ai/leaderboard/text/pareto?license=open-source
Llama3.1 instruct seems to be doing okay on that page, mostly because it's dirt cheap.
Re: Qwen 3.6 27B is the sweet spot for local development
#30The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...
But you have to factor in that this device will last you 5-10 years. That said, I wouldn't spend almost $7k USD on this macbook lol.