Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

621–630 of 848 posts

Re: Qwen 3.8 27B

#621

Post by throwaway27448 »

redox99 wrote: post #594
Wowfunhappy wrote: post #572

Earlier quoted context omitted.

> You just can't compress the entire human knowledge into a 30GB file. ...I'm just asking questions here... how sure are we of this? If you'd asked me six years ago whether we could compress all of human knowledge into a 1 TB file, I would have said no, and yet here we are. If you can do 1 TB, why not 30 GB? It's, like, within one order of magnitude.

(mumbles) Shannon entropy... Kolmogorov complexity... something, something... On a more serious note, it depends on your cutoff for "entire human knowledge". It's easy to prove for a generous interpretations of "entire human knowledge" that it can't be done, but hard for something like "all useful human knowledge".

Wikipedia (text) is about 25gigs compressed. I think that's a reasonable starting point.

Re: Qwen 3.8 27B

#622

Post by rasengan0 »

peri-cl wrote: post #27
KronisLV wrote: post #10

I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

Same here! Qwen3.6-35B-A3B is the only local model I've found that runs reasonably on my iGPU. Looks like me and and my noisily-wheezing laptop will be sitting out this upgrade.

Got Qwen3.8 to run on my Framework 12 Intel Core 13 Gen Raptor Lake i5-1334U small laptop with 48G RAM stick:

llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL

but it failed my basic prompt to compose a vim regex to match CamelCaseWords

downgraded a bit with Q4_K_M from ollama run qwen3.8:27b

and /set nothink and at least 1 regex matched FooBar

prompt eval at 2.8 t/s eval at 0.94t/s

I particularly enjoyed this usage of the regex: /%\1\%/ ... onward for 700+ characters of \%\/ :-)

Re: Qwen 3.8 27B

#623

Post by sbierwagen »

Wowfunhappy wrote: post #572
redox99 wrote: post #164

Earlier quoted context omitted.

Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.

> You just can't compress the entire human knowledge into a 30GB file. ...I'm just asking questions here... how sure are we of this? If you'd asked me six years ago whether we could compress all of human knowledge into a 1 TB file, I would have said no, and yet here we are. If you can do 1 TB, why not 30 GB? It's, like, within one order of magnitude.

Amusingly enough, someone from a frontier lab could probably answer this empirically. Years ago Microsoft demonstrated training LLMs on synthetic text-- books rewritten by LLMs to be more concise and more accurate. https://arxiv.org/abs/2306.11644 It is well known that Anthropic extensively uses synthetic text in training. You could probably get good 50tile, 90tile, 99 etc numbers just from the size of the training materials on Anthropic servers.

Re: Qwen 3.8 27B

#624

Post by fragmede »

MrDrMcCoy wrote: post #610
suprjami wrote: post #561

Earlier quoted context omitted.

Temperature, top-up, top-k, min-p all control which token the model predicts next and how likely it is to select one token over the other. You might understand this as "The capital of France is..." and the model isn't always going to select "Paris". Sometimes it will start a descriptive sentence or even get the answer wrong. That selection of the next token is what these settings control, and lots of sub-optimal sele…

I broadly knew that about temperature, but lack the background in machine learning/statistics to differentiate top-n-sigma from top-k/top-p.

So do I, but we live in the future: https://chatgpt.com/share/6a7fc3d2-39f4-83e8-a7c6-825ddfb5e7...

Re: Qwen 3.8 27B

#625

Post by MrDrMcCoy »

bilekas wrote: post #618
MrDrMcCoy wrote: post #606

Earlier quoted context omitted.

> Excuse me, but thats a direct link you've just sent. I asked where I can find the links. I like to believe in the source of truth. You asked where to find the GGUF files of this model for direct download and I provided it. Almost all useful model files that can be downloaded are hosted on Huggingface. > They shared a lot of links, I'm struggling to find yours. Where did yours come from ? I went to Huggingface, went…

> I went to Huggingface, went to the Unsloth org, as they tend to be the best, went to the Model page, and went to the "Files and versions" tab. "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh" Okay. > Unsloth AI is a very popular, highly reputable organization that takes upstream model files And How am I supposed to know that arriving to…

> "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh"

The path I described is all within HuggingFace.

> And How am I supposed to know that arriving to huggingface as a new user? Enlighten me.

Because I told you, knowing it was the best starting point for newbies.

> Cool.. Why don't they share em because I genuinely cant find em, I'm dumb.

You could go to Qwen's organization page on HuggingFace, it has a search function at the top, but you would be better served sticking with Unsloth.

> Well, I'm willing, but not from people who I might burn good will. Gracious. You do you.

Expecting others to do everything for you is not the same as trying things and asking questions about what you found.

Re: Qwen 3.8 27B

#626

Post by hirako2000 »

johnnyApplePRNG wrote: post #565
farlight wrote: post #504

Earlier quoted context omitted.

They were significantly more expensive for coding compared to DeepSeek's offering because of much higher cost of cache reads and lower cache hit rate. Now, after the price jump, the effective cost of deepseek.com API vs 3-rd party providers is about the same.

DeepSeek's API was never "cheap". They were stealing your code for that extremely cheap cached token price. That was never the real price. Every Chinese LLM API provider is the same to the best of my knowledge. "CHEAP API" (but pssst we train on every single input aka your code and ideas are ours forever)

There is no more, no less, guarantee that U.S providers don't do exactly that. In fact openAi is going to court.

If some models are cheaper it's simply because they cost far less to develop. And/or run.

Re: Qwen 3.8 27B

#627

Post by urbnspacecowboy »

CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

> Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks.

I don't expect you to blab publicly about your private benchmark, but what sorts of reasoning does it require?

Re: Qwen 3.8 27B

#628

Post by fragmede »

voiceeh wrote: post #505
johnnyApplePRNG wrote: post #306

Earlier quoted context omitted.

I honestly think we've got about a year tops.

I'd consider a year a while, but then, time is relative.

I've heard that some people even have a plan on how they'd like their life to look like in 5 years that they're aiming towards.

Re: Qwen 3.8 27B

#630

Post by searealist »

kimsey0 wrote: post #571

If anyone else is running this on an RTX 5090, https://github.com/Neroued/ninfer as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.

Just enable MTP on llama.cpp and you will get the same decode speeds.
Post reply on HN