Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

821–830 of 848 posts

Re: Qwen 3.8 27B

#821

Post by CMay »

m-s-y wrote: post #812
CMay wrote: post #657

Earlier quoted context omitted.

Well, I will say: #1: it does not require deep world knowledge, because that's not what local models are for. #2: it directly attacks drive-by understanding, overly linear processing training, poor attention mechanisms, poor reasoning patterns or lazy assumptions that ignore very easy low hanging fruit. #3: it requires solid instruction following in the face of errors. a lot of models will run into errors and then fa…

Are you willing to share this benchmark’s internals? Kinda weird to expect folks to take you at your word without the ability to “trust but verify”

The nature of LLM benchmarking is that they seem to saturate public benchmarks so quick, they are a uniquely efficient case of https://en.wikipedia.org/wiki/Goodhart%27s_law

I'm not asking anyone to take my word, they can believe or not and in practice people should be taking signals from a variety of places and doing their own testing to see how models behave in their own use cases. What I'm measuring and why I'm measuring it may not be the most important metric for your specific use case.

Most other models are simply failing at these tasks. I think the tasks are relevant to overall model capability, but they are not the only metric. You don't give a jellyfish a tool and expect it to produce wonders, so the other capabilities of the model matter.

Re: Qwen 3.8 27B

#822

Post by bmitc »

Retro_Dev wrote: post #816
bmitc wrote: post #656

Earlier quoted context omitted.

> correctly reason through one of my private benchmarks Want to say more about these private benchmarks? :)

seems to me like "private" is a good descriptor - I also have a set of "private" test cases - and they are kept private on purpose so they aren't scraped and fine-tuned on.

That clarifies it.

Re: Qwen 3.8 27B

#823

Post by suprjami »

Der_Einzige wrote: post #815
suprjami wrote: post #669

Earlier quoted context omitted.

Top-K: example setting 20. Select only from the 20 most likely tokens. Top-P: example setting 0.9. Select tokens whose probably accumulates to this number. So say you have tokens with 0.7 then 0.2 then 0.1, the last will not be selected because the first two tokens already accumulated to >=0.9. Min-P: example setting 0.05. Don't select tokens less probable than this value. So a token with 0.1 would be considered, a t…

Min-p is specifically "Don't select tokens less probable than a multiple of the top token's probability" with min_p of 0.1 multiplied by an example top probability of 0.3 being 0.03 as the truncation at that time step. Source: One of the min_p authors

Thanks, today I learned!

Re: Qwen 3.8 27B

#824

Post by gaigalas »

greenicon wrote: post #755
gaigalas wrote: post #413

Earlier quoted context omitted.

I don't see an MTP entry on Unsloth though. Maybe it's not available in a lower quant I need for my poor GPU.

It's included in the model gguf itself.

Maybe I'm not seeing the speed improvements because it's a dense model, and I'm comparing it to the 3.6 MoE with MTP. Likely, I'm attributing the speed I see to something else.

Honestly, if MTP is available, I don't know why it's so much slower here.

Re: Qwen 3.8 27B

#825

Post by fwipsy »

ElectricalUnion wrote: post #809
fwipsy wrote: post #671

Earlier quoted context omitted.

I'd like to see a human create a better pelican SVG without being able to look at the results. LLMs are language models, not (natively) vision models*. That's why it's a good benchmark: it engages LLM's logical reasoning in a way that we can check visually. The fact that they make errors which can be spotted visually, doesn't prove that LLMs are so far behind humans on language/logic tasks. Besides which, "fundamenta…

AFAIK the only well-known LLM with no "conventional" multimodal encoders required for audio and images is Gemma 4 12B.

Hmm, doesn't seem to translate to an especially coherent pelican: https://xcancel.com/TeksEdge/status/2063108620842356970

Re: Qwen 3.8 27B

#826

Post by anotherCodder »

anotherCodder wrote: post #817

vllm on 4x 5090 is getting ~20 tok/s with mtp on (their own thread on the hf card). i had qwen3.8-27b up the day after release, one rtx pro 6000, 140 tok/s spec, 0.156s first token, full 262k. image and video on the same api. numbers: https://github.com/avifenesh/memra try it: https://inference.tiyuvta.ai/app $0.38 in / $0.20 cache / $2.60 out. openrouter's only host right now is 23 tok/s at $0.45 / $3.20.

[dead]

Re: Qwen 3.8 27B

#827

Post by CamperBob2 »

acka wrote: post #428
CamperBob2 wrote: post #146

Earlier quoted context omitted.

I never install this stuff manually anymore. Just tell your LLM of choice to download model X from URL Y, build the latest inference engine of choice E, and then create batch files or shell scripts to run instruct and/or reasoning models in accordance with instructions at URL Z.

While you probably have the audacity to check the commands and code that your favorite LLM generates to complete this task, i'm pretty much certain that at some point in time, due to this pervasive mindset of "just let an AI agent do it," we'll have a front page news story of someone who had their family photos on their NAS uploaded to some public space, their security cameras connected to a "live porn for lolz" webr…

It's a risk, all right. I don't even use containers or VMs, I run automated installs on a dedicated server.

Very occasionally I'll have Claude install something on my dev box, but not without close supervision.

Re: Qwen 3.8 27B

#828

Post by zerologadmin »

jtrn wrote: post #771

I have been running it on my M5 Mac and was impressed with how well it worked with Pi coder. It can genuinely work as an assistant fully locally. It helps me configure Dockerfiles, fixed a couple of errors in a test Nuxt app, and so forth. Not very fast at 20 tps (8-bit quant for total memory usage around 30 GB), but enough to feel that I have a true local coding buddy. Then came the cold water shower. The agent kept…

[flagged]

Re: Qwen 3.8 27B

#829

Post by jmiskovic »

Der_Einzige wrote: post #180
skrebbel wrote: post #178

Earlier quoted context omitted.

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

Yes yes, oh god yes. They also spread FUD in the form of terrible recommended sampler settings. If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later.

The Qwen team published the same sampler settings for 3.8 and presumably they used those while testing on benchmark. Do you believe they could have achieved higher result with top-n-sigma?

Re: Qwen 3.8 27B

#830

Post by Zambyte »

mdp2021 wrote: post #789
croes wrote: post #739

Earlier quoted context omitted.

Models don’t know that they don’t know.

> know that they don’t know And we are waiting for architectures that do - because it's duly.

Honestly it seems like a job for the harness, rather than the model. Sample the model with the same question, perhaps with varying temperature (?), and use that to establish a degree of confidence in the answer. If the model provides very different answers every time, respond that it doesn't know. If it responds with the same answer usually but a different answer sometimes, respond with moderate confidence. If the model always responds with the same answer, respond with certainty.
Post reply on HN