Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

811–820 of 848 posts

Re: Qwen 3.8 27B

#811

Post by me_bx »

dannyw wrote: post #557
me_bx wrote: post #394

Earlier quoted context omitted.

In my experience with Qwen3.6 35B-A3B, disabling thinking made the model generate inaccurate replies. Ask it for the recipe of egg salad and it gives you the recipe of an omelette. Did I miss something, is it possible to have that model be reliable without thinking?

The official docs recommend different sampler options for thinking off. Have you tried adjusting those?

Thank you and sibling poster for the suggestions.

I used the settings recommended on huggingface/unsloth's model page + whatever suggestion from various LLMs - didn't research too much myself which setting did what.

Re: Qwen 3.8 27B

#812

Post by m-s-y »

CMay wrote: post #657
urbnspacecowboy wrote: post #627

Earlier quoted context omitted.

> Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. I don't expect you to blab publicly about your private benchmark, but what sorts of reasoning does it require?

Well, I will say: #1: it does not require deep world knowledge, because that's not what local models are for. #2: it directly attacks drive-by understanding, overly linear processing training, poor attention mechanisms, poor reasoning patterns or lazy assumptions that ignore very easy low hanging fruit. #3: it requires solid instruction following in the face of errors. a lot of models will run into errors and then fa…

Are you willing to share this benchmark’s internals? Kinda weird to expect folks to take you at your word without the ability to “trust but verify”

Re: Qwen 3.8 27B

#813

Post by nullbio »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

Ask it to do a zebra on a jetski to escape the benchmax.

Re: Qwen 3.8 27B

#814

Post by mycall »

qeternity wrote: post #729
WhyNotHugo wrote: post #718

Earlier quoted context omitted.

This is pretty much Chinese grammar with English words. In Chinese you'd say "need ask" or "I need ask" if you want to be unusually verbose, but there's no "to". All your examples align with this too. It just sounds to me like its optimising by removing the superfluous English words and thinking more Chinesely.

Has nothing to do with Chinese. Frontier labs have already been doing this for a while, verified in smuggled traces from OAT/Ant. Simply a way to reduce tokens.

https://github.com/juliusbrussee/caveman

Re: Qwen 3.8 27B

#815

Post by Der_Einzige »

suprjami wrote: post #669
MrDrMcCoy wrote: post #610

Earlier quoted context omitted.

I broadly knew that about temperature, but lack the background in machine learning/statistics to differentiate top-n-sigma from top-k/top-p.

Top-K: example setting 20. Select only from the 20 most likely tokens. Top-P: example setting 0.9. Select tokens whose probably accumulates to this number. So say you have tokens with 0.7 then 0.2 then 0.1, the last will not be selected because the first two tokens already accumulated to >=0.9. Min-P: example setting 0.05. Don't select tokens less probable than this value. So a token with 0.1 would be considered, a t…

Min-p is specifically "Don't select tokens less probable than a multiple of the top token's probability" with min_p of 0.1 multiplied by an example top probability of 0.3 being 0.03 as the truncation at that time step.

Source: One of the min_p authors

Re: Qwen 3.8 27B

#816

Post by Retro_Dev »

bmitc wrote: post #656
CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

> correctly reason through one of my private benchmarks Want to say more about these private benchmarks? :)

seems to me like "private" is a good descriptor - I also have a set of "private" test cases - and they are kept private on purpose so they aren't scraped and fine-tuned on.

Re: Qwen 3.8 27B

#817

Post by anotherCodder »

vllm on 4x 5090 is getting ~20 tok/s with mtp on (their own thread on the hf card). i had qwen3.8-27b up the day after release, one rtx pro 6000, 140 tok/s spec, 0.156s first token, full 262k. image and video on the same api. numbers: https://github.com/avifenesh/memra try it: https://inference.tiyuvta.ai/app $0.38 in / $0.20 cache / $2.60 out. openrouter's only host right now is 23 tok/s at $0.45 / $3.20.

Re: Qwen 3.8 27B

#818

Post by satvikpendem »

gpm wrote: post #507
satvikpendem wrote: post #485

Earlier quoted context omitted.

As I said to a dead reply, for coding all of that is immaterial, as long as it codes well then that's all that matters to most people, except it seems those who have an idelogical issue in which case the other model companies also have issues.

No, coding does not make ethical issues immaterial. Nor does it justify your behavior of harassing someone to do something that they've clearly decided not to do for ethical reasons on the basis of criteria that they have clearly rejected.

"Harassing" lol by telling them to try a model, alright. Anyway I didn't know they didn't want to use it for ideological reasons, they didn't say anything about that. Now that I know there's nothing more to be said.

Re: Qwen 3.8 27B

#819

Post by satvikpendem »

virgildotcodes wrote: post #532
satvikpendem wrote: post #485

Earlier quoted context omitted.

As I said to a dead reply, for coding all of that is immaterial, as long as it codes well then that's all that matters to most people, except it seems those who have an idelogical issue in which case the other model companies also have issues.

The funny thing is it doesn’t even code well. It’s still worse than OpenAI, Anthropic, and the leading open source models. So you get worse outcomes while also financially supporting one of the most prolific pushers of race wars, the destruction of labor rights, environmental protections, and general misinformation. Truly a win-win? Contrary to your framing it seems to only reason to use Grok at this point would be i…

It's a better deal compared to those is what I said. It codes well enough so that's why people use it. As I said, go to r/cursor and you'll find loads of people using it while OpenAI and Anthropic are more expensive per token and give less usage.

Re: Qwen 3.8 27B

#820

Post by satvikpendem »

InsideOutSanta wrote: post #524
satvikpendem wrote: post #485

Earlier quoted context omitted.

As I said to a dead reply, for coding all of that is immaterial, as long as it codes well then that's all that matters to most people, except it seems those who have an idelogical issue in which case the other model companies also have issues.

I don't care how good Grok is, it will be a cold day in hell when I give a guy who goes to AfD rallies, instrumentally supported the current Ebola outbreak, and gives the Hitler salute on stage (don't try to tell me he did not, I saw the video) any money I'm not absolutely forced to. And yes, OpenAI and Anthropic are highly questionable, too, and so I've ended my OpenAI subscription and I'm moving from Claude to othe…

Great, don't use it then.
Post reply on HN