Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unsloth Dynamic 3.0 GGUFs

unsloth.ai

Post reply on HN
Print view View original

61–70 of 125 posts

Re: Unsloth Dynamic 3.0 GGUFs

#61

Post by m463 »

walrus01 wrote: post #50

It would be nice if unsloth published GGUFs would use a version number or something, because now I have multiple different files on local storage that otherwise have exactly the same name. "Qwen3.8-27B-UD-Q8_K_XL.gguf" for instance. The one downloaded at least 4 days ago is a different thing and is NOT the "Dynamic 3.0" GGUF which I am now downloading, which I presume will have a different sha256 checksum? The unslot…

I wonder if there should be some kind of header sort of like a README inside mixed with something like EXIF information that camera images embed.

Re: Unsloth Dynamic 3.0 GGUFs

#62

Post by walrus01 »

jlbprof wrote: post #60
walrus01 wrote: post #50

It would be nice if unsloth published GGUFs would use a version number or something, because now I have multiple different files on local storage that otherwise have exactly the same name. "Qwen3.8-27B-UD-Q8_K_XL.gguf" for instance. The one downloaded at least 4 days ago is a different thing and is NOT the "Dynamic 3.0" GGUF which I am now downloading, which I presume will have a different sha256 checksum? The unslot…

How can I tell which one I have? I dl'd mine a few days ago.

I have just run a sha256 checksum on both copies now, the one I downloaded 4-5 days ago, and the one that's on the unsloth huggingface page released today, and will see if they're the same or different.

(downloaded in the last few hours after the announcement of dynamic 3.0)

Qwen3.8-27B-UD-Q8_K_XL-unsloth-dynamic3.0$ openssl dgst -sha256 *.gguf

SHA2-256(Qwen3.8-27B-UD-Q8_K_XL.gguf)= af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377

=====separator=========

downloaded at least 4 days ago:

Qwen3.8-27B-UD-Q8_K_XL-unsloth-original$ openssl dgst -sha256 Qwen3.8-27B-UD-Q8_K_XL.gguf

SHA2-256(Qwen3.8-27B-UD-Q8_K_XL.gguf)= af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377

So they're actually the same thing, but the announcement says released today... Please let's not confuse the end users any more than they already are.

Re: Unsloth Dynamic 3.0 GGUFs

#63

Post by walrus01 »

m463 wrote: post #61
walrus01 wrote: post #50

It would be nice if unsloth published GGUFs would use a version number or something, because now I have multiple different files on local storage that otherwise have exactly the same name. "Qwen3.8-27B-UD-Q8_K_XL.gguf" for instance. The one downloaded at least 4 days ago is a different thing and is NOT the "Dynamic 3.0" GGUF which I am now downloading, which I presume will have a different sha256 checksum? The unslot…

I wonder if there should be some kind of header sort of like a README inside mixed with something like EXIF information that camera images embed.

there's metadata in a GGUF (as can be seen when loading one into llama-server with verbose level 4) but it doesn't appear to show what the expected sha256 of the file is.

Re: Unsloth Dynamic 3.0 GGUFs

#64

Post by gunalx »

walrus01 wrote: post #63
m463 wrote: post #61

Earlier quoted context omitted.

I wonder if there should be some kind of header sort of like a README inside mixed with something like EXIF information that camera images embed.

there's metadata in a GGUF (as can be seen when loading one into llama-server with verbose level 4) but it doesn't appear to show what the expected sha256 of the file is.

Well that would not work as the metadata is also part of the sha, and if you have a sha predictably contain or hash to itself you have a bad hashing algorithm. (Unless of course you mean sha of the weigths and not the gguf itself)

Re: Unsloth Dynamic 3.0 GGUFs

#65

Post by prima-facie »

walrus01 wrote: post #50

It would be nice if unsloth published GGUFs would use a version number or something, because now I have multiple different files on local storage that otherwise have exactly the same name. "Qwen3.8-27B-UD-Q8_K_XL.gguf" for instance. The one downloaded at least 4 days ago is a different thing and is NOT the "Dynamic 3.0" GGUF which I am now downloading, which I presume will have a different sha256 checksum? The unslot…

You can manage these easily with the huggingface python app.

hf download hf://unsloth/Qwen3.8-27B-GGUF \ Qwen3.8-27B-UD-Q4_K_XL.gguf

and then see them with `hf cache ls`.

Prune old versions with `hf cache prune`.

Re: Unsloth Dynamic 3.0 GGUFs

#66

Post by CMay »

walrus01 wrote: post #62
jlbprof wrote: post #60

Earlier quoted context omitted.

How can I tell which one I have? I dl'd mine a few days ago.

I have just run a sha256 checksum on both copies now, the one I downloaded 4-5 days ago, and the one that's on the unsloth huggingface page released today, and will see if they're the same or different. (downloaded in the last few hours after the announcement of dynamic 3.0) Qwen3.8-27B-UD-Q8_K_XL-unsloth-dynamic3.0$ openssl dgst -sha256 *.gguf SHA2-256(Qwen3.8-27B-UD-Q8_K_XL.gguf)= af36ecb6b5db1407953345b746c14ac93f…

I think they said they were keeping the old UD 2.0 quant for the larger sizes? So maybe they kept those the same and simply reuploaded them. They said the newer UD 3.0 quant performed worse on some things for the higher quants. So now it's a mix of UD 3.0 and UD 2.0.

Re: Unsloth Dynamic 3.0 GGUFs

#67

Post by zozbot234 »

sosodev wrote: post #46
josh-wrale wrote: post #31

Earlier quoted context omitted.

Multiple agents need tokens. Should optimize for that instead of one agent blocking the others.

One agent typically blocks the others on a local device because the GPU is already completely utilized either in terms of memory or compute. You can have true parallelism at home, but you need an absurd amount of resources. It's not a simple threading problem.

The typical bottleneck to wider batching on consumer hardware is memory capacity for the KV-cache, not compute (even unified memory/iGPU-based platforms have enough compute to allow for some batching, and SSD offloading changes the scenario entirely). Qwen models tend to have bulky KV-caches for any given token count. But agentic swarms might end up sharing a large cache prefix, so there's scope for potential gains there.

Re: Unsloth Dynamic 3.0 GGUFs

#70

Post by redox99 »

sosodev wrote: post #46
josh-wrale wrote: post #31

Earlier quoted context omitted.

Multiple agents need tokens. Should optimize for that instead of one agent blocking the others.

One agent typically blocks the others on a local device because the GPU is already completely utilized either in terms of memory or compute. You can have true parallelism at home, but you need an absurd amount of resources. It's not a simple threading problem.

I have no problem running two or three sequences of qwen 27B with a 3090. It's basically the recommended way, LLM inference without batching is super inefficient.
Post reply on HN