Extremely cool!
Can't wait to give it a spin with ollama, if ollama could list it as a model that would be helpful.
Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
31–40 of 181 posts
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#32Very cool and works pretty well!
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#33How do I run this on Android?
Pocket Pal is what I've seen used before. Although recently heard about "Off Grid" but not read any reviews about it or tried it personally so caveat emptor. Will see if the community has other suggestions
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#341 bit with a FP16 scale factor every 128 bits. Fascinating that this works so well. I tried a few things with it. Got it driving Cursor, which in itself was impressive - it handled some tool usage. Via cursor I had it generate a few web page tests. On a monte carlo simulation of pi, it got the logic correct but failed to build an interface to start the test. Requesting changes mostly worked, but left over some symbol…
[dead]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#35Very cool and works pretty well!
I'm fascinated by these smaller models.
The amount of progress they've been making is incredible.
Is anyone following this space more closely? Is anyone predicting performance at certain parameter sizes will plateau soon?
Unlike the frontier models, these don't seem to be showing much progress of slowing down.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#36[dead]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#37Doesn't Jevons paradox dictate larger 1-bit models?
Yeah, hopefully they release >100B models.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#38It's been a hell of a morning for llama heads - first this, then the claude drop and turboquant.
I'm currently setting this one up, if it works well with a custom LoRa ontop ill be able to run two at once for my custom memory management system :D
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#39I can’t see how this is possible. You’re losing so much information.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#40[flagged]