Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Laguna S 2.1

poolside.ai

Post reply on HN
Print view View original

91–98 of 98 posts

Re: Laguna S 2.1

#91

Post by Lwerewolf »

Lwerewolf wrote: post #89
embedding-shape wrote: post #86

Earlier quoted context omitted.

More updates from the Discord (also mostly about the NVFP4): > we've got an NVFP4 checkpoint that has a KD of 0.135 relative to the BF16 checkpoint (for W4A4, on some GSM8K prompts). [...] We're still seeing some looping on W4A4 (although much less than before), whereas for W4A16 we've not been able to make it loop. Given that W4A4 is still a little broken, we won't make this an official release (we have one more ide…

Tool calling in pi completely broke with the updated q4 gguf (spinquant-less). Guessing it'll take some time.

...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real.

Re: Laguna S 2.1

#92

Post by ilc »

Lwerewolf wrote: post #40
sosodev wrote: post #33

Earlier quoted context omitted.

What inference server are you using? They have a custom branch for llama.cpp, but I wouldn't be surprised at all if it still needs fixing.

This: https://github.com/Blaizzy/mlx-lm/tree/pc/add-lg ...and this is what I should probably wait for (not sure why it's in vlm): https://github.com/Blaizzy/mlx-vlm/tree/pc/laguna-s-nvfp4 ...or perhaps I should've just used the gguf with the provided llama.cpp instead of trying to run the nvfp4-mlx from the get go, but where's the chaos in that :) Running deepseek flash on something locally now, this will have to wai…

This is definitely one where wait 2-3 weeks and look again, feels like the right strategy. It could be good, it could not. It is too volatile to say.

Re: Laguna S 2.1

#93

Post by dofm »

yogeshp wrote: post #29
mft_ wrote: post #23

Looks impressive, and this size fits achievable home hardware. That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.) Edit: someone in the process of doing so: https://huggingfa…

They have also published smaller 33B model called Laguna XS 2.1, its Q4 gguf is 20GB. https://huggingface.co/poolside/Laguna-XS-2.1-GGUF/tree/main

The XS model is genuinely confounding. So far I do not know what to make of it.

Maybe the chat template is broken, maybe not. But I can't get it to think/reason, or at least sometimes it seems to decide not to even when asked. And it is wildly inconsistent when it does it, sometimes thinking out loud.

However, without thinking enabled it is shockingly good at researching using web tool calls. It's also very fast.

I am assuming something is not yet right with the chat template, even using a recent build of llama.cpp, and I guess I will revisit this model again at some point in the future.

Re: Laguna S 2.1

#94

Post by Lwerewolf »

ilc wrote: post #92
Lwerewolf wrote: post #40

Earlier quoted context omitted.

This: https://github.com/Blaizzy/mlx-lm/tree/pc/add-lg ...and this is what I should probably wait for (not sure why it's in vlm): https://github.com/Blaizzy/mlx-vlm/tree/pc/laguna-s-nvfp4 ...or perhaps I should've just used the gguf with the provided llama.cpp instead of trying to run the nvfp4-mlx from the get go, but where's the chaos in that :) Running deepseek flash on something locally now, this will have to wai…

This is definitely one where wait 2-3 weeks and look again, feels like the right strategy. It could be good, it could not. It is too volatile to say.

Might wanna try it now, seems to have been largely fixed. Check huggingface threads and reddit. Works for me, very memory hungry and PP speed drops off a cliff around 200k context (~30tok/s decode and ~40tok/s pp - like... hope it isn't debugging with big logs), but it brings a fresh perspective alongside ds4-flash. The more, the merrier - I prefer keen eyeballs more than speed anyways.

Re: Laguna S 2.1

#95

Post by kevincox »

This seems like a great option for local usage. With 16 GiB of VRAM on the official (https://huggingface.co/poolside/Laguna-S-2.1-GGUF) Q4_K_M quant (which seems to be recommended) I can get about 50 t/s in and 10 t/s out. Definitely not winning any speed benchmarks but totally usable for background tasks.

The intelligent and decision making looks good overall. It seems to roughly compare to unsloth/Qwen3.6-27B-GGUF:UD-Q8_K_XL while being way faster on my hardware. It does often get stuck considering the same decision over and over again but maybe this is how it makes the better decisions.

However it does seem to have a bad habit of corrupting things (most often my name). It will rewrite kevincox to kevinc or kevcox or kevancox. Then fail to read directories or rewrite code with the wrong string and mess it up. Then it can't identify it's mistake.

Maybe I'll have try try out the Q8 model, but the description seems to push away from this one.

Re: Laguna S 2.1

#96

Post by mvallebr »

I am SO impressed by this model! It's cheap as deepseek v4 flash, better than minimax m3 I would say, and almost a glm 5.2. 120b params only for such a good result, with 258k context. This is amazing.

Yesterday I was making experiments running laguna xs 2.1 in my local machine (a smaller version), with a ryzen 5700x, 64 RAM and a 4060 ti 8gb nvidia

It was able to run locally with a context of 128k using llama-cpp and I tried using it for local development in a project that used uv, python, tool calls, file system explorations, websearches - all worked as a charm... in my own hardware

I was SO impressed. Then I saw open router is letting people try laguna S 2.1 for free - it's a model with 120b params and 258k context - if you top up at least 10 usd, it gives you 1000 requests per day for free so you can try it out. but the paid version is super cheap as well, specially the cache.

Re: Laguna S 2.1

#97

Post by alfiedotwtf »

mvallebr wrote: post #96

I am SO impressed by this model! It's cheap as deepseek v4 flash, better than minimax m3 I would say, and almost a glm 5.2. 120b params only for such a good result, with 258k context. This is amazing. Yesterday I was making experiments running laguna xs 2.1 in my local machine (a smaller version), with a ryzen 5700x, 64 RAM and a 4060 ti 8gb nvidia It was able to run locally with a context of 128k using llama-cpp and…

Yep! I loved Qwen 3.5 and 3.6, jumped to Minimax 2.5 and 2.7, but Laguna S2.1 feels like I have SOTA at home.

We’re going to get to the point where the enormous size of models will not be able to keep being updated with the huge amount of content being generated online, and so I think a strong SOTA-esque thinking model with a good harness, fast web crawler, and large MCP capabilities will be the future tool of choice, rather than larger and larger static models (unless someone creates upgradable but compact embeddings)

Re: Laguna S 2.1

#98

Post by bitexploder »

Archit3ch wrote: post #47
mft_ wrote: post #23

Looks impressive, and this size fits achievable home hardware. That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.) Edit: someone in the process of doing so: https://huggingfa…

> if someone would kindly quantise this down for the 64GB paupers The Q4_K_M is 75GB. I'm exactly at 64GB, and I wouldn't quantize it further. Instead, do partial weight residency and stream the rest from SSD.

I am going to see what I can get out of this streaming from SSD vs. Qwen 27B. Laguna is a seriously impressive model. I already have a lot of success with 27B A3B.
Post reply on HN