Interesting post.
Curious to know how they arrived at intelligence density = Negative log of the model's error rate divided by the model size.
Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
51–60 of 181 posts
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#52[deleted]
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#53Oh, boy.
This good tool hates my LM Studio...
The following message appears when I run Bonsai in my LM Studio.
I think my settings have done something wrong.
```
Failed to load the model
Error loading model.
(Exit code: null). Please check the settings and try loading the model again.
```
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#54Interesting post. Curious to know how they arrived at intelligence density = Negative log of the model's error rate divided by the model size.
What is model's error rate?
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#55What is the value of a 1 bit? For those that do not kno
I can port it to an FPGA and so can you.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#56I’m really curious how this scales up. Bonsai delivers an 8B model in 1.15 GB. How large would a 27B or 35B model be? Would it still retain the accuracy of those large models? If the scaling holds, we could see 100+B models in 64 GB of RAM.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#57I can’t see how this is possible. You’re losing so much information.
It's because they're natively trained with 1 bit, so it's not losing anything. Now, the question might be how they manage to get decent predictive performance with such little precision. That I don't know.
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#58Earlier quoted context omitted.
Thanks for sharing the link to your instance. Was blazing fast in responding. Tried throwing a few things at it with the following results: 1. Generating an R script to take a city and country name and finding it's lat/long and mapping it using ggmaps. Generated a pretty decent script (could be more optimal but impressive for the model size) with warnings about using geojson if possible 2. Generate a latex script to…
> Was blazing fast in responding. I should note this is running on an RTX 6000 pro, so it's probably at the max speed you'll get for "consumer" hardware.
consumer hardware?
That... pft. Nevermind, I'm just jealous
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#59Oh, boy. This good tool hates my LM Studio... The following message appears when I run Bonsai in my LM Studio. I think my settings have done something wrong. ``` Failed to load the model Error loading model. (Exit code: null). Please check the settings and try loading the model again. ```
It needs a mlx fork because the lowest bit in mlx is 2 currently (for affine quantization).
Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
#60Earlier quoted context omitted.
> Was blazing fast in responding. I should note this is running on an RTX 6000 pro, so it's probably at the max speed you'll get for "consumer" hardware.
consumer hardware ? That... pft. Nevermind, I'm just jealous
Look it was my present to myself after the Figma IPO (worked there 5 years). If you want to feel less jealous, look at the stock price since then.