Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

prismml.com

Post reply on HN
Print view View original

111–120 of 181 posts

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#111

Post by newman314 »

simonw wrote: post #82

You can run this model on an iPhone via the latest update to this Locally AI app: https://apps.apple.com/us/app/locally-ai-local-ai-chat/id674... For its size (1.2GB download) it's very impressive. Here's a pelican it drew me running on my phone - the SVG comments are good, the image not so much: https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D...

One thing I discovered tonight is that it appears smaller models are remarkably bad at converting time between timezones.

I tested the following using almost all available models on Locally and did not get a single model that got the right answer.

"What is 9:30 am (Taiwan Standard Time, TST) in US Pacific?"

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#112

Post by AnthonBerg »

jjcm wrote: post #16

1 bit with a FP16 scale factor every 128 bits. Fascinating that this works so well. I tried a few things with it. Got it driving Cursor, which in itself was impressive - it handled some tool usage. Via cursor I had it generate a few web page tests. On a monte carlo simulation of pi, it got the logic correct but failed to build an interface to start the test. Requesting changes mostly worked, but left over some symbol…

As someone whose brain was addled by exposure to art history, I strongly support the suggested pelican on bicycle.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#113

Post by fxwin »

I'm very skeptical of the advantage they're claiming here. The whitepaper [0] only compares these to full precision models, when the more interesting (and probably more meaningful) comparison would be with other quantized models with a similar memory footprint.

Especially considering that these models seem to more or less just be quantized variants of Qwen3 with custom kernels and other inference optimizations (?) rather than fine tuned or trained from scratch with a new architecture, I am very surprised (or suspicious rather) that they didn't do the obvious comparison with a quantized Qwen3.

Their (to my knowledge) new measure/definition of intelligence seems reasonable, but introducing something like this without thorough benchmarking + model comparison is even more of a red flag to me.

[0] https://github.com/PrismML-Eng/Bonsai-demo/blob/main/1-bit-b...

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#115

Post by riedel »

fxwin wrote: post #113

I'm very skeptical of the advantage they're claiming here. The whitepaper [0] only compares these to full precision models, when the more interesting (and probably more meaningful) comparison would be with other quantized models with a similar memory footprint. Especially considering that these models seem to more or less just be quantized variants of Qwen3 with custom kernels and other inference optimizations (?) ra…

Actually IMHO the promise would be beyond standard FP4 quants. I think the goal is more where 1.58 bit (ternary) quants are heading. Having said that it would be interesting to see performance on nonstandard HW.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#116

Post by Imustaskforhelp »

freakynit wrote: post #93

Open access for next 5 hours (8GiB model, running on RTX 3090) or until server crashes or the this spot instance gets taken away :) => https://ofo1j9j6qh20a8-80.proxy.runpod.net ./build/bin/llama-server \ -m ../Bonsai-8B.gguf \ -ngl 999 \ --flash-attn on \ --host 0.0.0.0 \ --port 80 \ --ctx-size 65500 \ --batch-size 512 \ --ubatch-size 512 \ --parallel 5 \ --cont-batching \ --threads 8 \ --threads-batch 8 \ --cache-t…

Kind sir, May I say to you thanks for doing so! I really appreciate it :D

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#117

Post by kgeist »

freakynit wrote: post #109
txrx0000 wrote: post #69

Earlier quoted context omitted.

I always remind myself and everyone else that human DNA is "only" 1.6 GB of data, and yet it encodes all of the complex systems of the human body including the brain, and can replicate itself. Our intuitive feel of how much stuff can be packed into how many bits are probably way off from the true limits of physics.

Human DNA has 3.2 billion base pairs, and with 2x the information density compared to binary systems (due to 4-letters as opposed 2), that's roughly 800MB of informational data. Second, what's even more crazy is that roughly 98% of that DNA is actually non-coding.. just junk. So, we are talking about encoding entirety of the logic to construct a human body in just around 16MB of data!!! That's some crazy levels of re…

>Second, what's even more crazy is that roughly 98% of that DNA is actually non-coding.. just junk.

I think it's a myth that non-coding DNA is junk. Say:

https://www.nature.com/articles/444130a

>'Non-coding' DNA may organize brain cell connections.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#118

Post by nl »

I ran my custom agentic SQL debugging benchmark against it and I'm impressed.

Results: 8 passed, 0 failed, 17 errored out of 25

That puts it right between Qwen3.5-4B (7/25) and Nanbeige4.1-3B (9/25) for example, but it took only 200 seconds for the whole test. Qwen3.5 took 976 seconds and Nanbeige over 2000 (although both of these were on my 1070 so not quite the same hardware)

Granite 7B 4bit does the test in 199 seconds but only gets 4/25 correct.

See https://sql-benchmark.nicklothian.com/#all-data (click on the cells for the trace of each question)

Errors are bad tool calls (vs failures which is incorrect SQL)

I used @freakynit's runpod (thanks!)

[1] https://news.ycombinator.com/item?id=47597268

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#119

Post by IshKebab »

voxelghost wrote: post #88
simonw wrote: post #82

You can run this model on an iPhone via the latest update to this Locally AI app: https://apps.apple.com/us/app/locally-ai-local-ai-chat/id674... For its size (1.2GB download) it's very impressive. Here's a pelican it drew me running on my phone - the SVG comments are good, the image not so much: https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D...

Did you ask for a pelican with a bicycle, or was that just an added bonus?

It's a well known LLM test. Google "SVG pelican bicycle".

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#120

Post by algoth1 »

pdyc wrote: post #62
jjcm wrote: post #16

1 bit with a FP16 scale factor every 128 bits. Fascinating that this works so well. I tried a few things with it. Got it driving Cursor, which in itself was impressive - it handled some tool usage. Via cursor I had it generate a few web page tests. On a monte carlo simulation of pi, it got the logic correct but failed to build an interface to start the test. Requesting changes mostly worked, but left over some symbol…

thanks, i tested it, failed in strawberry test. qwen 3.5 0.8B with similar size passes it and is far more usable.

Does asking it to think step by step, or character by character, improves the answer? It might be a tokenization+unawareness of its own tokenization shortcomings
Post reply on HN