Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

prismml.com

Post reply on HN
Print view View original

131–140 of 181 posts

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#131

Post by nl »

Imustaskforhelp wrote: post #124
nl wrote: post #118

I ran my custom agentic SQL debugging benchmark against it and I'm impressed. Results: 8 passed, 0 failed, 17 errored out of 25 That puts it right between Qwen3.5-4B (7/25) and Nanbeige4.1-3B (9/25) for example, but it took only 200 seconds for the whole test. Qwen3.5 took 976 seconds and Nanbeige over 2000 (although both of these were on my 1070 so not quite the same hardware) Granite 7B 4bit does the test in 199 se…

I have been using @freakynit's runpod as well all be it, I like making working pomodoro apps as my own custom test, and although its not good for it (none of the prototypes work), I feel like it can be good within a specific context like Sql as you mention. I imagine this being used as sub-agents with some sota models directing them but I wasn't really able to replicate it personally (I had asked Claude to create a d…

> I feel like it can be good within a specific context like Sql as you mention.

Yes I think very constrained task: known data universe, well known language etc should be the best possible place for small language models to play

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#132

Post by croemer »

simian1983 wrote: post #5

Ran this prompt: Share an interesting scientific discovery or historical fact that most people don't know about. Please explain why it's significant and its impact on our understanding. Got this nonsense answer: Interesting Scientific Discovery: The Discovery of the "Boson" in 1924 by Charles K. Alton ("Chuck") Alton What it is: In 1924, American physicist Charles K. Alton discovered the "boson", a particle that woul…

[dead]

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#133

Post by freakynit »

freakynit wrote: post #93

Open access for next 5 hours (8GiB model, running on RTX 3090) or until server crashes or the this spot instance gets taken away :) => https://ofo1j9j6qh20a8-80.proxy.runpod.net ./build/bin/llama-server \ -m ../Bonsai-8B.gguf \ -ngl 999 \ --flash-attn on \ --host 0.0.0.0 \ --port 80 \ --ctx-size 65500 \ --batch-size 512 \ --ubatch-size 512 \ --parallel 5 \ --cont-batching \ --threads 8 \ --threads-batch 8 \ --cache-t…

Update: this has been evicted by runpod as it was on spot.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#135

Post by Imustaskforhelp »

nl wrote: post #131
Imustaskforhelp wrote: post #124

Earlier quoted context omitted.

I have been using @freakynit's runpod as well all be it, I like making working pomodoro apps as my own custom test, and although its not good for it (none of the prototypes work), I feel like it can be good within a specific context like Sql as you mention. I imagine this being used as sub-agents with some sota models directing them but I wasn't really able to replicate it personally (I had asked Claude to create a d…

> I feel like it can be good within a specific context like Sql as you mention. Yes I think very constrained task: known data universe, well known language etc should be the best possible place for small language models to play

Yes, I think though that, maybe 1) this shows 1-bit llm models working so more companies can do that so that we can get more competition within this space (+ ngram-embedding idea)

Another point, but I feel like, we can see some really good fine tuned models out of this model, the community feels excited about 1-bit LLM architecture. We are gonna see some good innovation within this space in the upcoming future.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#136

Post by simonw »

voxelghost wrote: post #88
simonw wrote: post #82

You can run this model on an iPhone via the latest update to this Locally AI app: https://apps.apple.com/us/app/locally-ai-local-ai-chat/id674... For its size (1.2GB download) it's very impressive. Here's a pelican it drew me running on my phone - the SVG comments are good, the image not so much: https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D...

Did you ask for a pelican with a bicycle, or was that just an added bonus?

The prompt I always use for this is:

  Generate an SVG of a pelican riding a bicycle

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#138

Post by adrian_b »

freakynit wrote: post #109
txrx0000 wrote: post #69

Earlier quoted context omitted.

I always remind myself and everyone else that human DNA is "only" 1.6 GB of data, and yet it encodes all of the complex systems of the human body including the brain, and can replicate itself. Our intuitive feel of how much stuff can be packed into how many bits are probably way off from the true limits of physics.

Human DNA has 3.2 billion base pairs, and with 2x the information density compared to binary systems (due to 4-letters as opposed 2), that's roughly 800MB of informational data. Second, what's even more crazy is that roughly 98% of that DNA is actually non-coding.. just junk. So, we are talking about encoding entirety of the logic to construct a human body in just around 16MB of data!!! That's some crazy levels of re…

As another poster has said, much of the "junk" is not junk.

The parts of the DNA with known functions encode either proteins or RNA molecules, being templates for their synthesis.

The parts with unknown functions include some amount of true junk caused by various historical accidents that have been replicated continuously until now, but they also include a lot of DNA that seems to have a role in controlling how the protein or RNA genes are expressed (i.e. turning off or on the synthesis of specific proteins or RNAs), by mechanisms not well understood yet.

Re: Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs

#140

Post by adrian_b »

humanjir wrote: post #100
txrx0000 wrote: post #69

Earlier quoted context omitted.

I always remind myself and everyone else that human DNA is "only" 1.6 GB of data, and yet it encodes all of the complex systems of the human body including the brain, and can replicate itself. Our intuitive feel of how much stuff can be packed into how many bits are probably way off from the true limits of physics.

That's not strictly true - DNA doesnt replicate itself, a cell with DNA replicates itself. You need to count the information contained in the non-DNA part of the cell too. Just in case it's not obvious, you can't take human DNA and put it in a cat cell, it won't work, that cell won't replicate.

True.

For now, the DNA replication and the synthesis of RNA and proteins using the information stored in DNA are the best understood parts about how a cell grows and divides, but how other complex cellular structures, e.g. membranes or non-ribosomal peptides, are assembled and replicated is much less understood.

We need more years of research, perhaps up to a decade or two, until we will be able to know the entire amount of information describing a simple bacterial cell, and perhaps more than that for a much more complex eukaryotic cell.

Post reply on HN