Great! I hope the era of 1-bit LLMs really gets going.
Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
151–160 of 181 posts
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#152[dead]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#153I can’t see how this is possible. You’re losing so much information.
I always remind myself and everyone else that human DNA is "only" 1.6 GB of data, and yet it encodes all of the complex systems of the human body including the brain, and can replicate itself. Our intuitive feel of how much stuff can be packed into how many bits are probably way off from the true limits of physics.
It encodes the data on top of locally optimal trajectories in the physical world that were learned in millions of years of evolution. Treat this as context, not weights.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#154Oh, boy. This good tool hates my LM Studio... The following message appears when I run Bonsai in my LM Studio. I think my settings have done something wrong. ``` Failed to load the model Error loading model. (Exit code: null). Please check the settings and try loading the model again. ```
It needs a mlx fork because the lowest bit in mlx is 2 currently (for affine quantization).
That mlx is for apple hardware only, though? Or did I misunderstand something.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#155[flagged]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#156[flagged]
Think the commercially viable comment is a reference to the license not technical characteristics
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#157A bit misleading to say they take 14x less memory, no one is doing inference with 16-bit models.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#158I’m really curious how this scales up. Bonsai delivers an 8B model in 1.15 GB. How large would a 27B or 35B model be? Would it still retain the accuracy of those large models? If the scaling holds, we could see 100+B models in 64 GB of RAM.
[dead]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#159[dead]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#160[flagged]
llm/bot comment