21–30 of 59 posts
A benchmark table comparing to Qwen 3.5 35B-A3B seems strange when Qwen 3.6 35B-A3B has been out for some time and is significantly better. I didn't notice the version difference when first reading the article! So this is a heads up to people like me.
I wish that "small" LLMs would stop being confidently very incorrect. Admittedly this is a bit of an intentionally esoteric test, but the confident way in which it presents a totally incorrect answer is a bit concerning. "please write 250 words on the etymology and history of the word schlong" https://pastes.io/uhshFgn4 The actual origin of the word is from middle high German and Yiddish-speaking Ashkenazi Jewish com…
Would it not be better to ask models to search the topic on the Internet and then answer? I do not understand why we expect small LLMs to answer from own knowledge.
It's also something I've seen has great results with esoteric individual pieces of knowledge that works fine in a Q6 or Q8 quantized LLM but breaks down in a bad way at worse quantization.
I wish that "small" LLMs would stop being confidently very incorrect. Admittedly this is a bit of an intentionally esoteric test, but the confident way in which it presents a totally incorrect answer is a bit concerning. "please write 250 words on the etymology and history of the word schlong" https://pastes.io/uhshFgn4 The actual origin of the word is from middle high German and Yiddish-speaking Ashkenazi Jewish com…
I wish that "small" LLMs would stop being confidently very incorrect. Admittedly this is a bit of an intentionally esoteric test, but the confident way in which it presents a totally incorrect answer is a bit concerning. "please write 250 words on the etymology and history of the word schlong" https://pastes.io/uhshFgn4 The actual origin of the word is from middle high German and Yiddish-speaking Ashkenazi Jewish com…
The schlong test is nearly as funny as drawing shit on bicycles test
Very excited to see how it performs, I’ve been a bit skeptical of the efficacy of converting existing models - really cool to see one trained from scratch in the ternary format.
Also I love AI sites. Fancy font, plain serious style, we "introduce" rather than "release". It's an AI not an animal after all.
A benchmark table comparing to Qwen 3.5 35B-A3B seems strange when Qwen 3.6 35B-A3B has been out for some time and is significantly better. I didn't notice the version difference when first reading the article! So this is a heads up to people like me.
Their main comparison is 1-bit Bonsai 27B (Qwen3.6 27B) which beats A3B anyway.
Really basic stuff. But then again, the entire thing was running in But before anyone says 1-bit bonsai 27B beats anything, please actually run it and ask it some questions about topics you already know the answer to.
While Qwen 3.6 35B A3B in Q8 with full context capability (llama-server in no-mmap mode with 262k context will eat 47GB, so not comparable in size either) knows a great deal. The 35B-A3B can even translate multiple pages of English into Farsi and its Farsi output is not far off the quality of what Google Translate does.
I haven't tested something as badly quantized as 35B A3B Q2 which is somewhere around 12GB on disk. https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
Earlier quoted context omitted.
Their main comparison is 1-bit Bonsai 27B (Qwen3.6 27B) which beats A3B anyway.
Beats how? In my experience 1-bit bonsai 27B is quite "dumb" when asked a question about a lot of things, as a canned repository of static knowledge from its training dataset. I mean, I literally asked it for a 250 word description of Seattle and it hallucinated a tallest building in the city with an observation deck that doesn't exist, and didn't mention the Space Needle. Really basic stuff. But then again, the enti…
Earlier quoted context omitted.
Beats how? In my experience 1-bit bonsai 27B is quite "dumb" when asked a question about a lot of things, as a canned repository of static knowledge from its training dataset. I mean, I literally asked it for a 250 word description of Seattle and it hallucinated a tallest building in the city with an observation deck that doesn't exist, and didn't mention the Space Needle. Really basic stuff. But then again, the enti…
these tools are for manipulating and retrieving and transforming sequences of context. using it as a knowledgebase is just expecting the wrong thing.