Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Liquid AI reveals 8B-A1B MoE trained on 38T

liquid.ai

Post reply on HN
Print view View original

91–100 of 101 posts

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#91

Post by sheeshkebab »

satvikpendem wrote: post #58
walrus01 wrote: post #54

Earlier quoted context omitted.

I personally find any model smaller than something like Qwen 3.6 35B-A3B (8-bit quantization, about 49GB memory usage when loaded into llama.cpp) to be too "stupid" for reliable use. I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience o…

Qwen 3.6 27B dense is much better than the 35B MoE model for coding, not sure if you've tried that yet.

27b is slow as molasses vs 35b on local stuff I have (m5 max). Mtp doesn’t make any difference either.

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#92

Post by hgoel »

andai wrote: post #33
Ifkaluva wrote: post #29

Liquid does amazing work, but I kinda feel like they are overtraining their models. 38T tokens seems like a lot for an 8B model

What's the downside? Don't they stop when they hit diminishing returns?

Wouldn't the model start overfitting at some point? Degrading generalization for accuracy on the training set.

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#93

Post by alfiedotwtf »

npodbielski wrote: post #84
intothemild wrote: post #78

Earlier quoted context omitted.

You should enable MTP now that its available. LLamaCPP has had some massive updates in the last week or so.

Yes, Qwen 3.6 MoE is hitting like 80-90tk/s on Strix halo. On R9700 I had like 170t/s. It was not possible to keep up. But MoE is circling very often. I switch then to dense model and have 20-30t/s but it is able to solve quite a lot of tasks.

For those speeds, I’m assuming Q4?

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#94

Post by alfiedotwtf »

theanonymousone wrote: post #68
walrus01 wrote: post #54

Earlier quoted context omitted.

I personally find any model smaller than something like Qwen 3.6 35B-A3B (8-bit quantization, about 49GB memory usage when loaded into llama.cpp) to be too "stupid" for reliable use. I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience o…

Have you seen the 8bit quantisation matter a lot? The "consensus" in r/LocalLlama is that up to 4 bits the loss is tolerable.

It’s not a general rule, and depends highly on the model and the quantisation used. Don’t guess, Unsloth sometimes publish graphs in their tutorials showing the error rate vs file size… sometimes Q4 is great, other times I go for Q6

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#95

Post by onlyrealcuzzo »

h14h wrote: post #87
onlyrealcuzzo wrote: post #40

I just tested this on a bug fixing benchmark I'm working on. It did not perform as well as I expected. Qwen2.5-Coder-3B (2 years old) outperformed it by a wide range -> fixing ~50% of bugs whereas this model only fixed ~12%. Granted, it's not a coder specific model, but given its benchmark performance to Gemma models, and that it's two years newer, and that it's an MoE with 8B total params, I expected it to be more c…

That's not all that surprising, IMO. From what I understand, LiquidAI is focusing pretty narrowly on building models that operate as the "agentic core" of a larger system. If I were going to use this model, I'd be looking to use it more as is the primary chat interface of a larger system, and having it orchestrate & delegate tasks to other places via tool calls. It's not quite as exciting on the surface as a local "d…

Interesting...

Two of the other responses speak about it being abysmal at tool calling.

Overall, I'm pretty impressed a model this small can find/fix ~12% of bugs with crappy context - even if they're about as easy as possible to fix.

I just assumed it would perform better, given all the advancements in the space.

It's possible 1B active parameters is just not enough - even if it has 8B params of knowledge to reason through bugs.

Playing around with the context I fed it, it was able to fix up to ~34% of bugs vs ~46% for Qwen2.5-Coder-3B and ~54% for Qwen2.5-Coder-7B.

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#96

Post by riversflow »

ahoka wrote: post #79
kennywinker wrote: post #32

Earlier quoted context omitted.

That’s not what’s happening. The question is revealing that the model has a model of language but not of reality. It knows what words go together, but not real-world concepts.

This. LLMs are marketed on the false premise of all knowledge, intelligence and wisdom being possible to be encoded in language only.

lol, i think the LLM shows more wisdom here than the average person. Functionally, being 50m away from the car wash is at the car wash if you have a dirty car in your possession that needs cleaning. Realistically, the only reason you express the need to go to the carwash if you are in a 50m proximity with your car you intend to clean at the carwash is if you need to walk in and talk to someone.

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#97

Post by walrus01 »

theanonymousone wrote: post #77
walrus01 wrote: post #70

Earlier quoted context omitted.

Absolutely. Difference in Q6 vs Q8 is not as immediately noticeable, but if I test by starting from a blank slate context and giving it the same complicated task with Q4 vs a Q8 GGUF file loaded, the difference is apparent. The Q4 will struggle or do 'stupid' things with even simple bash or python. Q4 might not be as noticeable for conversational purely text one on one interaction with an LLM, but when you dig deeper…

Thanks a lot. How about Q8 vs FP16/BF16? Have you checked them too?

I have not spent a lot of time running FP16 'full precision' versions of some things, but as the other commenter says, it's not much difference. There's a really wide array of benchmarks and tests from a lot of third parties unrelated to the trainer of the AI models that shows at most a two percent difference in score and capability between BF16 and Q8.

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#98

Post by HappMacDonald »

roenxi wrote: post #89
andai wrote: post #57

Earlier quoted context omitted.

“Car Wash” test with 53 models https://news.ycombinator.com/item?id=47128138 This article has a graph of the human response rates. About 70% correct on average. Accuracy depends on the country (maybe a language barrier?). See also original thread on the car wash thing. I want to wash my car. The car wash is 50 meters away. Should I walk or drive? https://news.ycombinator.com/item?id=47031580

"Correct" is pushing it, the question is too vague if approached as a genuine question and not a gotcha. I've actually had literal experiences where I wanted to wash my car and walked to a car wash in the past. That was me collecting the car, and there is an argument that would be a valid walk answer. If we require logical rigour there isn't enough context in the question. If we allow for informal language then there…

Well presumably it's simply better for the environment and for your own health to just carry the car with you

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#99

Post by npodbielski »

alfiedotwtf wrote: post #93
npodbielski wrote: post #84

Earlier quoted context omitted.

Yes, Qwen 3.6 MoE is hitting like 80-90tk/s on Strix halo. On R9700 I had like 170t/s. It was not possible to keep up. But MoE is circling very often. I switch then to dense model and have 20-30t/s but it is able to solve quite a lot of tasks.

For those speeds, I’m assuming Q4?

Ud_Q4_k_xl

Re: Liquid AI reveals 8B-A1B MoE trained on 38T

#100

Post by Woodi »

2001zhaozhao wrote: post #67

At some point we have to be running into some inherent mathematical limits of knowledge compression, right? No way the knowledge benchmarks on these 8B models will keep getting better without overfitting on these benchmarks

Yea, it's strange all that all possible books stealing movement and then lobbying for law prohibiting... something.

Humans train "thinking methodology" first and then know how to use it while accessing data and to build knowledge.

Humans do not memorize at once all text in existence, that's totally stupid.

Already thinking humans specialize in disciplines: math, chemistry, IT, cooking, etc while still using new data.

All of that computing is local- on the LAN of the brain.

So if some "agents" wants to help then there is zero need for computation outside of home/corporation/car local area network.

Licenses ??

Post reply on HN