Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

281–290 of 848 posts

Re: Qwen 3.8 27B

#281

Post by dd8601fn »

mixologic wrote: post #236
tinfoilhatter wrote: post #196

Earlier quoted context omitted.

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

Nothing absurd about that. What do you think an "Executive producer" is? A "Director" ? Does Peter Jackson get credit for creating the Lord of the Rings Trilogy films? Christopher Nolan for his films? But did he make them ? No, it was the collective effort of thousands of individuals all working under their direction. Just like if somebody creates software today, and the end result is generated by the collective effo…

That’s an interesting way to look at it, and still generous to the llm.

By which I mean it puts the llm in the role of actor and such… people practicing an art. It seems harder to make a case that the llm is actually making creative decisions rather than aping a synthesis of past human decisions and expressions of taste, strictly, at every turn.

Though I guess it’s all “shades of grey”. I never felt like I was contributing a ton when I was essentially wrapping a large, capable library that some smarter person or people made, either.

Tangentially… remember back when we used to see headlines like “12 year old made a web browser!” and we all thought, “No. No they didn’t.”

It all feels a little like that, again.

Re: Qwen 3.8 27B

#282

Post by cjbprime »

hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Hm, I have a 4090 as well, and:

$ build/bin/llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off

0.02.993.689 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 911.53 MiB on device 0: cudaMalloc failed: out of memory

Update: Oh, it works after I stop Xorg. But nvidia-smi only showed Xorg using 200M out of the 24G, so why would a 911M alloc fail?

Re: Qwen 3.8 27B

#283

Post by tinfoilhatter »

rob wrote: post #215
tinfoilhatter wrote: post #196

Earlier quoted context omitted.

So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.

I just read through a couple of your posts that weren't dead or buried, and it seems like you're pretty anti-AI. You should really start to have an open mind towards it. It's going to be the future (if it isn't already), and as you continue to get older, you're going to really wish you spent your time right now learning and embracing the technology instead of being so against it. A lot of the skills and things that y…

I'm not sure that typing messages to a chat bot requires much catching up, but since you think this is what I should do rob, I better listen!

Re: Qwen 3.8 27B

#284

Post by Jcowell »

cjbprime wrote: post #282
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Hm, I have a 4090 as well, and: $ build/bin/llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off 0.02.993.689…

There has to be a better way of sharing config , a profile of sorts you can upload and share

Re: Qwen 3.8 27B

#285

Post by oezi »

satvikpendem wrote: post #242
altruios wrote: post #235

Earlier quoted context omitted.

[flagged]

Yeah I'm sure everyone on r/cursor or in previous HN threads about Grok 4.5 or 4.6 are all unserious and insane. No one actually cares about the politics as long as the model codes well. Edit, quite interesting to see the reception to this comment compared to essentially the same type of comment I made on a Grok 4.6 benchmark HN post: https://news.ycombinator.com/item?id=49275385#49275571 It's true that Cursor gives…

I would say it is sad that there are people who use Grok when there are so many other choices available which don't come with the issues of supporting Musk.

It is not all just 'politics'. Take a stand on some issues. It doesn't cost much not to use Grok.

Re: Qwen 3.8 27B

#286

Post by Aurornis »

cjbprime wrote: post #282
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Hm, I have a 4090 as well, and: $ build/bin/llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off 0.02.993.689…

The person above is running the OS from a separate GPU, so the command can use all of the VRAM of the 4090.

If you're trying to use the 4090 for your desktop at the same time, you will need smaller context size. You could remove the '-c 170000' and try turning fit on to see what fits.

Re: Qwen 3.8 27B

#287

Post by bredren »

jwr wrote: post #127
TomGarden wrote: post #24

Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.

I have an M4 Max (unfortunately 64GB). I have been running the Qwen 35B A3B one for a while now, after testing and benchmarking a number of models. That one was consistently the best in class for tasks like despamming, E-mail classification, OCR and dictation post-processing. It was also really fast (90 tokens/s). I'm benchmarking the 3.8 model now, it seems it is better (near-perfect score on my E-mail spam filterin…

How are you using email despamming?

I am looking to automatically recover mail from junk folder due to bad icloud mail filtering.

Re: Qwen 3.8 27B

#288

Post by kennywinker »

segmondy wrote: post #194
Almondsetat wrote: post #132

The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help

You don't need $10k to run DeepSeek, I run it on a $1000 system.

Go on… what is this $1000 system?

Re: Qwen 3.8 27B

#289

Post by literoldolphin »

ferrouswheel wrote: post #268
literoldolphin wrote: post #211

Why is anyone even using video cards these days? You may as well be burning cash. This is the perfect candidate for just splattering it on your nvme and then reading it off there and into memory. All of these run perfectly fine on simple m4 silicone: https://github.com/drumih/turbo-fieldfare https://github.com/leonickson1/Swiftlet https://github.com/sqliteai/warp

Lol, "burn money on apple hardware instead!"

And yet it's also a laptop you can basically take anywhere unlike a giant video card with 1000 watt power supply requirements.

Re: Qwen 3.8 27B

#290

Post by freehorse »

bmitc wrote: post #138
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Lol at that command. Why is this stuff so hard to run locally? I've spent a few days trying to figure it all out and haven't been able to. LM Studio doesn't work behind proxies. Ollama is confusing and doesn't seem to support Qwen3? And Llama.cpp is your command. I just want to run ` ` with some default parameters set and for it to run locally.

If you do not want to tweak parameters you can just run

    llama-server -m model.gguf
That's it
Post reply on HN