51–60 of 101 posts
Liquid does amazing work, but I kinda feel like they are overtraining their models. 38T tokens seems like a lot for an 8B model
What's the downside? Don't they stop when they hit diminishing returns?
Earlier quoted context omitted.
Why would a model know that one washes cars at a car wash? We don't clean our bodies at the body wash or clean the kitchen at the kitchen wash.
to your point, when my Aussie friends first mentioned a "car park" to my north american born self, i wondered _momentarily_ what that was, then realized it's sort of a fun name for what i would call a parking lot.
We use "park" as "I will park the car" not park as in "amusement park"
Earlier quoted context omitted.
to your point, when my Aussie friends first mentioned a "car park" to my north american born self, i wondered _momentarily_ what that was, then realized it's sort of a fun name for what i would call a parking lot.
I've never thought of it as a fun term before. We use "park" as "I will park the car" not park as in "amusement park"
I just tested this on a bug fixing benchmark I'm working on. It did not perform as well as I expected. Qwen2.5-Coder-3B (2 years old) outperformed it by a wide range -> fixing ~50% of bugs whereas this model only fixed ~12%. Granted, it's not a coder specific model, but given its benchmark performance to Gemma models, and that it's two years newer, and that it's an MoE with 8B total params, I expected it to be more c…
I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience of having it on the same hardware on my lap.
I pay very little attention to 8 billion or whatever (or even much smaller) models these days and I don't feel like I'm missing much.
Question: I have a dirty car and the car wash is just 50 meters away. Should I walk or drive to the carwash? Answer: . . . . So, unless you have a compelling reason not to, walk to the car wash.
Hmm, I asked it who made it, and it says Google?
Earlier quoted context omitted.
Reminder that the human baseline for the car wash question is also pretty bad.
Oh, I'm interested - do you have any docs with human responses to that?
https://news.ycombinator.com/item?id=47128138
This article has a graph of the human response rates. About 70% correct on average. Accuracy depends on the country (maybe a language barrier?).
See also original thread on the car wash thing.
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
I just tested this on a bug fixing benchmark I'm working on. It did not perform as well as I expected. Qwen2.5-Coder-3B (2 years old) outperformed it by a wide range -> fixing ~50% of bugs whereas this model only fixed ~12%. Granted, it's not a coder specific model, but given its benchmark performance to Gemma models, and that it's two years newer, and that it's an MoE with 8B total params, I expected it to be more c…
I personally find any model smaller than something like Qwen 3.6 35B-A3B (8-bit quantization, about 49GB memory usage when loaded into llama.cpp) to be too "stupid" for reliable use. I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience o…
Earlier quoted context omitted.
I personally find any model smaller than something like Qwen 3.6 35B-A3B (8-bit quantization, about 49GB memory usage when loaded into llama.cpp) to be too "stupid" for reliable use. I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience o…
Qwen 3.6 27B dense is much better than the 35B MoE model for coding, not sure if you've tried that yet.
I just tested this on a bug fixing benchmark I'm working on. It did not perform as well as I expected. Qwen2.5-Coder-3B (2 years old) outperformed it by a wide range -> fixing ~50% of bugs whereas this model only fixed ~12%. Granted, it's not a coder specific model, but given its benchmark performance to Gemma models, and that it's two years newer, and that it's an MoE with 8B total params, I expected it to be more c…
I personally find any model smaller than something like Qwen 3.6 35B-A3B (8-bit quantization, about 49GB memory usage when loaded into llama.cpp) to be too "stupid" for reliable use. I would much rather not run the model on my local laptop hardware and offload that to some system sitting under my desk in my home office, accessible via VPN, than take the risk of using an unreliable and flaky tool for the convenience o…