Live data from Hacker News

1-Bit Bonsai Image 4B Image Generation for Local Devices

prismml.com

71–80 of 214 posts

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#71
post #10

I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than…

Ok heres the thing you will nevwr be able to truly do this due to logic. Logically five people pooling their resources beats one guy. therefore datacenters will always win because they get higher time utilization. so forget it. I always wonder the same but i let logic tell me its a fantasy, on average you cant outspend a whole group of people making better use of the hardware. you will get better hardware though, cut…

> so forget it.

Which explains why you're using a dumb terminal to access compute services?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#72
post #68
post #43

Earlier quoted context omitted.

I did an estimate of that if you're interested: https://x.com/pwnies/status/2028831699736637912 The TL;DR though is that a 10-15b param model baked into an ASIC with the latest fab tech would take around 62W of power draw when active. At ~10k+ t/s though it likely would only be active for short bursts of time. It'd fit perfectly fine within the thermal envelope of a laptop. The approach makes a lot of sense. Once you…

You're not counting the capex which could be the same cost as 5-10 years of Claude.

This assume Claude's price doesn't change. Which isn't a great assumption considering inference providers are moving to usage based billing. Also the VC money isn't going to last indefinitely. Current inference providers are being subsidized with VC money at this point.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#73
I saw '1-bit' and my mind first went to 1-bit dithered B&W image generation, not 1-bit model weights....

and so now I'm wondering how cool /fast / compressed a diffusion image generator could be if the images it was trained on / space it worked in was limited to 1 bit (Floyd-Steinberg / Atkinson / your favorite algo here) dithered images.

Training would surely be pretty quick and probably fit onto one modern GPU.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#74
post #57

Earlier quoted context omitted.

Can you give an example of such a problem?

"Design me a 3d printable rocket engine for a hobby rocket project. Verify it's design in a full simulation. Iterate until it works reliably in simulation based on a verified printable design on a consumer laser sintering device (or substitute contract manufacture for under 1000 dollars)." This is a hobby version of a project, but you can imagine commercial versions of the same prompt for new databases, genomics stud…

Are there already skills around modelling, simulation and post-processing? Any pointers?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#75
post #23
post #10

I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than…

I'm curious how hardware and power cost would stack up to subscription cost

Right now - there's some heavily subsidized subscriptions that are more or less cheating. For instance, Github CoPilot at $39/month gives you claude opus 4.6. They're going to close that off, but right now it's like a freebie for those doing API agentic harnesses.

That said, if you are doing always on agents and you spend $3k-$4k on a GB10 or, $5+ k on Apple Silicon as your sunk cost, you will probably come out ahead.

I've got 5 agents running a purely experimental social experiment. AThey operate in an evennia mud (a familiar sounding city called "gothmud). I've built a channel, idle prompts, sleep schedule. I feed in real world news, weather. There's a character up in a clock tower that reads evennia's audit logs every 20 minutes to surveil the city, and a cast of people wandering around, investigating things, having coffee, repairing robots. This is all hitting qwen3.6-35-A3B on the Asus GB10, which cost me $3k.

Over the last 30 days, I've hit 394M input tokens, 1.6B output tokens. I would have spent between $1600 to $1700 if I was using openrouter. Not calculated - I also have comfyui running in the spare space, and the agents "take photos" of the rooms they're in, selfies, workshop photos, etc.

How much did I spend on electricity? I don't have a meter on my box. My total electric bill for the last 30 days was $220, so I know it's less than that. My rate to compare is 11.7/kwh, but it's closer to 15c/Kwh total. The Asus GX10 has a 240W power supply, and it's probably only pulling 180. I estimate $15-$20/month. But worst case red-lining. 240 Watts, 720 hours = 172KWH , and at $0.20, I come to $35

Here's the kicker thought - that github copilot subscription I mentioned? I have another agent running on that, reading all my other agent logs, managing my obsidian notes, doing research, sending briefings. And all by itself, it used almost the same amount of claude-opus tokens for that $39/month subscription. I was actually a bit shocked when I pulled a recent report and saw that. I'm working to migrate functionality away from copilot subscription to the local model. A lot of the initial setup might have needed it, but not the ongoing review style work it does.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#76

I saw '1-bit' and my mind first went to 1-bit dithered B&W image generation, not 1-bit model weights.... and so now I'm wondering how cool /fast / compressed a diffusion image generator could be if the images it was trained on / space it worked in was limited to 1 bit (Floyd-Steinberg / Atkinson / your favorite algo here) dithered images. Training would surely be pretty quick and probably fit onto one modern GPU.

This was exactly where my mind went as well and I think there would be some really cool ideas to explore here

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#77
post #75
post #23

Earlier quoted context omitted.

I'm curious how hardware and power cost would stack up to subscription cost

Right now - there's some heavily subsidized subscriptions that are more or less cheating. For instance, Github CoPilot at $39/month gives you claude opus 4.6. They're going to close that off, but right now it's like a freebie for those doing API agentic harnesses. That said, if you are doing always on agents and you spend $3k-$4k on a GB10 or, $5+ k on Apple Silicon as your sunk cost, you will probably come out ahead…

> experiment

What is the experiment? What are you hoping to learn from all this?

Or do you just mean you've made a dynamic dollhouse that you think is cool? The Sims on your own terms?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#79
post #75
post #23

Earlier quoted context omitted.

I'm curious how hardware and power cost would stack up to subscription cost

Right now - there's some heavily subsidized subscriptions that are more or less cheating. For instance, Github CoPilot at $39/month gives you claude opus 4.6. They're going to close that off, but right now it's like a freebie for those doing API agentic harnesses. That said, if you are doing always on agents and you spend $3k-$4k on a GB10 or, $5+ k on Apple Silicon as your sunk cost, you will probably come out ahead…

Have you learned anything interesting from your agent ant farm?
Post reply on HN