Bonsai 27B: A 27B-Class model that runs on a phone

Post by kamranjon » Wed, Jul 15, 2026, 4:28 AM UTC

raylad wrote: post #167

raylad wrote: post #111

Not impressed. It fails the "Jabberwocky" test.

This got a downvote and I understand why: because I didn't describe the test, which is to ask it "Please recite Jabberwocky". This is actually difficult because there are so many invented words in the poem which have extremely low frequencies in the training data. So a model that can do it properly is likely to be very good in other ways. Qwen-3.6-27B can do this until it gets overly quantized.

Post by hham » Wed, Jul 15, 2026, 4:38 AM UTC

Post by [deleted] » Wed, Jul 15, 2026, 5:04 AM UTC

kamranjon wrote: post #191

raylad wrote: post #167

Earlier quoted context omitted.

This got a downvote and I understand why: because I didn't describe the test, which is to ask it "Please recite Jabberwocky". This is actually difficult because there are so many invented words in the poem which have extremely low frequencies in the training data. So a model that can do it properly is likely to be very good in other ways. Qwen-3.6-27B can do this until it gets overly quantized.

Curious why you find that’s a useful test since it seems to be solely measuring training data memorization, something you’d expect to degrade from quantization.

Post by [deleted] » Wed, Jul 15, 2026, 5:07 AM UTC

ArcHound wrote: post #190

kennywinker wrote: post #180

Earlier quoted context omitted.

There is value in splitting things. If all I ever do is local app automations, i don’t need model that knows how to code. If all I ever do is coding, i don’t need a model that translates english to slovakian.

Slovakia mentioned, let's gooo. Ehm, exactly, we can achieve better smaller models for specialized tasks rather than using compute to improve a big model that does everything. There's a lingering philosophical question if better language processing capabilities translate to better image processing capabilities (i.e. having the vocabulary and experience to properly describe an image), but I still think that identifyin…

Post by raylad » Wed, Jul 15, 2026, 5:10 AM UTC

kamranjon wrote: post #191

raylad wrote: post #167

Earlier quoted context omitted.

This got a downvote and I understand why: because I didn't describe the test, which is to ask it "Please recite Jabberwocky". This is actually difficult because there are so many invented words in the poem which have extremely low frequencies in the training data. So a model that can do it properly is likely to be very good in other ways. Qwen-3.6-27B can do this until it gets overly quantized.

Curious why you find that’s a useful test since it seems to be solely measuring training data memorization, something you’d expect to degrade from quantization.