Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

801–809 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#801

Post by amatecha »

Tenoke wrote: post #639
amatecha wrote: post #545

Earlier quoted context omitted.

"macOS" (or however they spell it now) is pretty bad, but I'm not sure it's possible Apple could ever possibly produce an OS as bad as Windows 11 lol, it's really surprising to me to see someone suggest it's somehow actually worse?! How many times has an Apple OS wiped your hard drive or otherwise been completely borked from a forced update? I know multiple people personally who have experienced this with Windows 10/…

>How many times has an Apple OS wiped your hard drive or otherwise been completely borked from a forced update I use Windows and this has never happened to me. I have had Macbooks I cant open to fix/replace something trivial while I can replace any part easily on a Windows PC/laptop though.

Right, that was a rhetorical question, highlighting the fact that such an occurrence is even possible, to such a degree of incidence that I know multiple people who have experienced it. Meanwhile, never once heard an anecdote about a MacOS update (though I can imagine it has happened to someone out there, just never heard about it-- in contrast to numerous news articles about Windows' frequently-destructive updates)

Re: Qwen 3.6 27B is the sweet spot for local development

#802

Post by SwellJoe »

lambda wrote: post #797
SwellJoe wrote: post #768

Earlier quoted context omitted.

Gemma 12B? It's unique in the Gemma family, and unique among vision models. It's a novel encoder-less model...the whole model is vision. Somehow. I don't understand it, but it blows away Gemma 4 31B and Qwen 27B in my tests. It's not even close. And, is also tiny and fast, compared to those larger models, so it's better and faster and smaller. Weird combo.

Tried it out. I'm compring against Qwen 3.5 122B-A10B, so a much larger model. It gets some correct, but Qwen 3.5 122B-A10B has done much better. Gemma 4 12B even hallucinated some species in trying to identify a plant, and the other guesses it made weren't all that close, while Qwen 3.5 122B-A10B got it right on the first try. 12B did get one right that 31B got wrong. I'd have to do a much more thorough eval to real…

Ah, yeah, a model ten times as large on disk will know more, for sure. My tasks are more about "what's happening in this image?" rather than "what is this thing?", which doesn't require encyclopedic knowledge, it need good reasoning about what it's seeing.

Re: Qwen 3.6 27B is the sweet spot for local development

#803

Post by pizza234 »

sanderjd wrote: post #285
wahnfrieden wrote: post #160

Earlier quoted context omitted.

Difficult... and wastefully expensive

Seems like an investment into building expertise, which is likely to have high ROI in the future, rather than a wasteful cost.

> Seems like an investment into building expertise

Learning LLM internals != using them as daily tools (the latter was the topic of discussion).

Shifting to the learning topic: no need for expensive hardware. Even a 16 GB GPU will do, and that's (relatively) cheap and available.

Re: Qwen 3.6 27B is the sweet spot for local development

#804

Post by pizza234 »

sanderjd wrote: post #281
pizza234 wrote: post #122

Earlier quoted context omitted.

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. Cloud models are (much) faster, they don't consume so much power/generate heat, they have much bigger (LLM) context, they're much more precise and they have a much wider (engineering) context of the given problem. Except privacy and use cases that are blocked by cloud models (e.g. reverse engineering), local LLMs are cur…

> currently The interesting question is whether that gap will narrow, and if so, how much, and on what timescale. The exact answer to this question is not knowable, but if you are the kind of person who comes to a site called "hacker news", and you think there is a nonzero chance that the answer is that yes, the gap will narrow and this won't always be an expensive toy, then now seems like a pretty great time to get…

The same applies to ray tracing; the question whether shadows can be rendered accurately and cheaply is also not knowable - but it's very unlikely that ray tracing will become as cheap as rasterization for the same quality target. LLMs are pretty much the same; they're inherently computationally expensive, even if optimizations are in continuous development.

Re: Qwen 3.6 27B is the sweet spot for local development

#805

Post by cpburns2009 »

Der_Einzige wrote: post #779
cpburns2009 wrote: post #465

Earlier quoted context omitted.

Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.

Please switch to using the far superior reptation penalty, DRY. It's built into llamacpp.

What are good DRY setting

Re: Qwen 3.6 27B is the sweet spot for local development

#806

Post by real-hacker »

onion2k wrote: post #20

None of the examples reflect 'real work', at least not what I'd consider real work. Being able to nail a zero-shot greenfield project is relatively easy even for a small model. There's not much context to build up and it can fall back to similar examples in the training data easily. So long as you're not asking it to invent something wholly new it'll probably manage. The real test is whether or not it can work with y…

The question has to be asked: is it (small models struggling at brownfield) an intelligence problem or a context problem? Even if it is an intelligence problem, is it possible to use a customized harness to achieve the same level of performance as SoTA models? That would be very valuable, I think.

Re: Qwen 3.6 27B is the sweet spot for local development

#807

Post by hollowturtle »

janalsncm wrote: post #752
hollowturtle wrote: post #630

Earlier quoted context omitted.

These people work mostly in CRUD apps and they're telling you they how feel productive. Btw exploratory ideas even for hard problems come out already after a hackaon of a day or a game jam of 3 days

Do you also believe Terrance Tao was a mediocre mathematician before AI?

What the hell has to do with spinning up PoC and web productivity?

Re: Qwen 3.6 27B is the sweet spot for local development

#808

Post by automajicly »

SwellJoe wrote: post #485
UncleOxidant wrote: post #464

Earlier quoted context omitted.

It would be great if the Gemma folks would release a code-focused model. Probably won't happen, but it's fun to dream.

The Ornith folks say they're doing that, but haven't released the Gemma-based 31b yet ( https://github.com/deepreinforce-ai/Ornith-1 ). But, also, the Qwen-based 35b MoE Ornith version performs worse than Qwen 3.6 and Qwen AgentWorld on my benchmarks (which are focused on finding security bugs, so not exactly the same as agentic coding, but closely related skills). That said, the reason they're able to release Ornith…

I know this wasn't meant as a response to me, but I thought I would chime in, if thats okay. Sorry its a bit of time since the timing of this message.And I hope Im not intrusive. But to give you a bit of an update, I broke down and subbed to Claude's PRO feature. I know it defeats the purpose of local and private, but I still have my Gemma in my private LM studio. WOW. The coding diff is like night and day. I had Claude Code audit my repo page and it was quick and production grade. Like it systematically just chiseld away at all the sloppy choppy and dead cade and even polished up the aesthetics of my page presentation. I need this as a local private model.

Re: Qwen 3.6 27B is the sweet spot for local development

#809

Post by devin »

simplyluke wrote: post #763
devin wrote: post #711

Earlier quoted context omitted.

I look forward to re-evaluating this statement in, what do you say, 12 months from now?

I’ll toss $10k in the s&p and you buy the rig and we’ll see who feels like they made a better call?

10k in the S&P is by default a far better investment than some computer components. You could say the same thing like "you buy a car and I'll put my money in the S&P and we'll see who's happier in N months". We were speculating on the cost of components going forward, not on whether the S&P is better place to park 10k.
Post reply on HN