Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1-Bit Bonsai Image 4B Image Generation for Local Devices

prismml.com

Post reply on HN
Print view View original

101–110 of 214 posts

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#101

Post by vineyardmike »

mirekrusin wrote: post #56
barnas2 wrote: post #34

Earlier quoted context omitted.

A company called Taalas is working on something like that. Not Opus4.6 quality, but I'm sure they're targeting larger models. Currently they're using a LLama 8B model. It runs at ~17k tokens per second, and you can test it at https://chatjimmy.ai/ .

It starts to be interesting when latency is better than average website.

I’m not sure if this is what you meant, but at 17k t/s, you start to compete with the speed of network calls. You could approach the point of generating an HTML/js/css page faster than some websites can be returned over the network.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#102

Post by IncreasePosts »

lwansbrough wrote: post #100

Can anyone think of any negative externalities of making generative photorealistic images illegal? I can think of a lot of positives. The negatives amount to a convoluted argument about the limits of free speech.

Prisoner 1: so, what are you in for?

Prisoner 2: I made a picture of a nice sunset over the ocean

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#104

Post by lwansbrough »

IncreasePosts wrote: post #102
lwansbrough wrote: post #100

Can anyone think of any negative externalities of making generative photorealistic images illegal? I can think of a lot of positives. The negatives amount to a convoluted argument about the limits of free speech.

Prisoner 1: so, what are you in for? Prisoner 2: I made a picture of a nice sunset over the ocean

If it were illegal it wouldn’t be readily available. You’d have to seek it out. People seeking it out wouldn’t be using it to generate a sunset.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#105

Post by hankbond »

sroussey wrote: post #99

I extracted the code from the web demo to add to make a web image generation node to my in browser ai workflow tool, and it’s pretty sweet. Waiting for xenova to add to transformersjs 4.3 and I’ll release as well. Couldn’t wait though to test.

can you describe your "in browser ai workflow tool"? I may or may not be working on something similar and am very interested in what others are building in the space.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#107

Post by Cider9986 »

flashman wrote: post #97

Twenty years ago, I don't think any of us were excited about a future internet where we couldn't trust whether what we were seeing or reading was genuine. I hope one day we'll be able to look back on this era as an aberration, like that scene in Mad Men where the Drapers fling their picnic rubbish onto the grass and drive away.

What could make it stop?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#108

Post by Obscurity4340 »

ytjohn wrote: post #85
swatcoder wrote: post #77

Earlier quoted context omitted.

> experiment What is the experiment? What are you hoping to learn from all this? Or do you just mean you've made a dynamic dollhouse that you think is cool? The Sims on your own terms?

A little of A, a little of B. I have a lot of fun building it out, it's surpassed Factorio in addiction, and I've been able to flesh out some patterns that I roll back into more productive agent harness bits. For A: The learning is in building agent harnesses that aren't just cron jobs reading a file like HEARTBEAT.md. I have some serious tools for my own use. One main assistant/coordinator agent, one SRE/coder agent…

Would you ever consider posrting a video of all this? It sounds equal parts delightful and terrifying

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#109

Post by windexh8er »

goofy_lemur wrote: post #46
lumost wrote: post #10

I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than…

Ok heres the thing you will nevwr be able to truly do this due to logic. Logically five people pooling their resources beats one guy. therefore datacenters will always win because they get higher time utilization. so forget it. I always wonder the same but i let logic tell me its a fantasy, on average you cant outspend a whole group of people making better use of the hardware. you will get better hardware though, cut…

Where I think you're wrong is that everything in technology has been cyclical, it's just a matter of time.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#110

Post by [deleted] »

woadwarrior01 wrote: post #53
moralestapia wrote: post #38

Earlier quoted context omitted.

Genuine question: doesn't it blow your mind that there exists a 1 Gigabyte file/program that can generate any image you can think of just from a rough description of it?

Where are you getting the 1 Gigabyte number from? Their 1-bit quantized Diffusion Transformer is just under 1 GB. You also need the text-encoder (4-bit quantized) and VAE (unquantized) for inference and their combined weight is ~3.42 GB. TBF, even at that size it's no less mind blowing.

[deleted]
Post reply on HN