Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

561–570 of 848 posts

Re: Qwen 3.8 27B

#561

Post by suprjami »

MrDrMcCoy wrote: post #473
Der_Einzige wrote: post #180

Earlier quoted context omitted.

Yes yes, oh god yes. They also spread FUD in the form of terrible recommended sampler settings. If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later.

For those us us who don't know, what do those parameters do and why are they better?

Temperature, top-up, top-k, min-p all control which token the model predicts next and how likely it is to select one token over the other.

You might understand this as "The capital of France is..." and the model isn't always going to select "Paris". Sometimes it will start a descriptive sentence or even get the answer wrong.

That selection of the next token is what these settings control, and lots of sub-optimal selections compound over time to produce a junk response.

Re: Qwen 3.8 27B

#562

Post by svdr »

Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?

Re: Qwen 3.8 27B

#563

Post by reilly3000 »

onlyrealcuzzo wrote: post #126
DanielHB wrote: post #51

Earlier quoted context omitted.

What kind of things you only get with a larger model?

Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming. I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated). What sounds good on paper - ofte…

I think someone ought to encode this into a harness. It is really insightful into how we should be spending time if it is going to be spent reviewing AI code.

Re: Qwen 3.8 27B

#564

Post by eblanshey »

hadlock wrote: post #406
eblanshey wrote: post #385

Earlier quoted context omitted.

Why not just run FP8 on vLLM with that much vRAM? It's plenty fast.

For high concurrency, using the blackwell's native native W4A4 MLP compute path, nvfp4 is something like a 1.2-1.5x performance increase over FP8. We're doing data enrichment (so, tasks completed successfully + tokens/second) so the performance bump shows up in the tasks/month number. I am just now getting the benchmarks running against 3.8 27b but I expect similar results from benching 3.6 27b at the same quant.

I see. Did you see any intelligence degradation between FP8 and NVFP4 for 3.6 27B? You're using vLLM, right?

Re: Qwen 3.8 27B

#565

Post by johnnyApplePRNG »

farlight wrote: post #504
johnnyApplePRNG wrote: post #401

Earlier quoted context omitted.

The price on DeepInfra and most providers on OpenRouter has not changed... if anything, it's gotten cheaper. All of this DeepSeek price increase stuff is related to their specific API, not the entire market.

They were significantly more expensive for coding compared to DeepSeek's offering because of much higher cost of cache reads and lower cache hit rate. Now, after the price jump, the effective cost of deepseek.com API vs 3-rd party providers is about the same.

DeepSeek's API was never "cheap".

They were stealing your code for that extremely cheap cached token price.

That was never the real price. Every Chinese LLM API provider is the same to the best of my knowledge.

"CHEAP API" (but pssst we train on every single input aka your code and ideas are ours forever)

Re: Qwen 3.8 27B

#566

Post by ethbr1 »

walrus01 wrote: post #522
pbhjpbhj wrote: post #521

Earlier quoted context omitted.

What are the chances that the colouring of actual pelicans is bleeding in to the prompt. Seems it could be that?

Doubtful? Without knowing anything about how it was trained, if you google image search "pelican bird" they are predominantly white in color. Though the bill is certainly orange, different shades depending on camera and lighting conditions. The SVG attempt looks more like an attempt at the product photos if you search "pelican case orange".

Need more tests of 'pelican bird standing in pelican equipment case' ;)

Re: Qwen 3.8 27B

#567

Post by anon373839 »

redox99 wrote: post #143
NitpickLawyer wrote: post #34

Earlier quoted context omitted.

> Beats Opus 4.7 Max I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage. Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expecta…

Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.

In my experience, the facts that are compressed away in small models are ones you don’t need them to memorize. They need familiarity with the essential concepts in a field, so that they will have better comprehension of material put into the context (and make better retrieval decisions). They don’t need to know very particular details which are a hallucination risk and should be derived from the context instead.

Re: Qwen 3.8 27B

#568

Post by XCSme »

shdh wrote: post #488
XCSme wrote: post #384

Earlier quoted context omitted.

5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama. I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com

How did you undervolt the 3090?

Msi afterburner, you go around 900mv curve editor, raise it up to normal clock frequency, and it uses like 260w instead 300w for same performance.

There's some YouTube guides for it.

I also undervolted my new 5070ti, same tdp, around 260w instead of 300w and like 8% better performance.

Re: Qwen 3.8 27B

#569

Post by horacemorace »

svdr wrote: post #562

Wow. This model is so good, and we have GLM 5.3 (seems great voor security related work) and Deepseek. In a few months we'll have Fable/Sol-like capabilities that are not coming from the big US companies. I feel as a programmer that that is more than enough. How wil OpenAI and Anthropic survive when frontier model intelligence becomes commoditized?

They’ll certainly try to stymie people by colluding with manufacturers until we get nvidia level hardware or LLM ASICs from the East.

Re: Qwen 3.8 27B

#570

Post by walrus01 »

ethbr1 wrote: post #566
walrus01 wrote: post #522

Earlier quoted context omitted.

Doubtful? Without knowing anything about how it was trained, if you google image search "pelican bird" they are predominantly white in color. Though the bill is certainly orange, different shades depending on camera and lighting conditions. The SVG attempt looks more like an attempt at the product photos if you search "pelican case orange".

Need more tests of 'pelican bird standing in pelican equipment case' ;)

Or to make it even more tricky, since Pelican cases will float when closed, something like "generate an SVG file of a Pelican bird floating in the ocean next to a Pelican case".
Post reply on HN