Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Laguna S 2.1

poolside.ai

Post reply on HN
Print view View original

61–70 of 98 posts

Re: Laguna S 2.1

#61

Post by nunodonato »

mft_ wrote: post #23

Looks impressive, and this size fits achievable home hardware. That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.) Edit: someone in the process of doing so: https://huggingfa…

unsloth is always here to save the day https://huggingface.co/unsloth/Laguna-S-2.1-GGUF

Re: Laguna S 2.1

#63

Post by aitchnyu »

megavon wrote: post #4

This is INSANE. How did they do this?

Are all AI labs in Google's weight class in crawling and ranking the Web's content? I know OpenAI has contractor subject matter experts in all topics.

Re: Laguna S 2.1

#64

Post by embedding-shape »

embedding-shape wrote: post #44
Lwerewolf wrote: post #24

Earlier quoted context omitted.

nvfp4 mlx, literally barebones pi. edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.

> edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings. Been playing around for a few hours with the poolside/Laguna-S-2.1-NVFP4 + poolside/Laguna-S-2.1-DFlash-NVFP4 + vLLM, been seeing the same behaviour. Usually new model releases are plagued with issues at release though, best to wait 1-2 weeks then retry, or better yet, investigate yourself :) Personally I haven't found any o…

Update: Seems quite literally they have bugs on the hardware I'm trying to run this with:

From Poolside CEO Eiso Kant on Twitter:

> Learning we have some bugs on the RTX6000. We’re on it. Team has worked non stop last days and it’s getting late for a lot of the inference folks, so might be until tomorrow till we have a solution. - https://x.com/eisokant/status/2079693050796785720

Update2: I'm now running poolside/Laguna-S-2.1-NVFP4 with vLLM 0.23.1rc1.dev1378+gd6dbdb9b0 (FlashInfer 0.6.14) and seeing slightly better results in regards to the looping. I can't see any specific changes that would affect this though, strangely enough.

Re: Laguna S 2.1

#65

Post by andy99 »

nunodonato wrote: post #61
mft_ wrote: post #23

Looks impressive, and this size fits achievable home hardware. That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.) Edit: someone in the process of doing so: https://huggingfa…

unsloth is always here to save the day https://huggingface.co/unsloth/Laguna-S-2.1-GGUF

Seems it’s not fully supported in mainline llama.cpp yet https://github.com/ggml-org/llama.cpp/pull/25165

In the huggingface link they mention building for CPU and for CUDA, does anyone know if that means it wouldn’t be possible to build targeting Vulkan?

Re: Laguna S 2.1

#68

Post by reindeer2 »

I've been following work on the second-order effects that ripple through the system for a while. This is the first treatment I've seen that the framing reveals an assumption that isn't explicitly defended.

Re: Laguna S 2.1

#69

Post by andy99 »

andy99 wrote: post #65
nunodonato wrote: post #61

Earlier quoted context omitted.

unsloth is always here to save the day https://huggingface.co/unsloth/Laguna-S-2.1-GGUF

Seems it’s not fully supported in mainline llama.cpp yet https://github.com/ggml-org/llama.cpp/pull/25165 In the huggingface link they mention building for CPU and for CUDA, does anyone know if that means it wouldn’t be possible to build targeting Vulkan?

Replying to myself, seems this PR was merged into main and it the model does work with a Vulkan backend on my Framework desktop, I’m getting about 220 tok/s prompt processing and 21 tok/s output on the 4-bit quant. This is really a sweet spot imo on this machine between maximizing ram use and still having decent speed due to the expert size. This looks really promising.
Post reply on HN