Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1-Bit Bonsai Image 4B Image Generation for Local Devices

prismml.com

Post reply on HN
Print view View original

191–200 of 214 posts

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#191

Post by jeroenhd »

mft_ wrote: post #145
fulafel wrote: post #120

Earlier quoted context omitted.

> Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. Not the bottom end - most people are on laptops or mobile devices that are much lower GPU power than this.

Probably the bottom end an individual would want to consider using due to slow generation time . Sure, you could theoretically take a model compressed in this manner and deploy it on an old netbook and run the calculations on the CPU, but each image would probably take an hour…

My laptop has a Pascal-era Nvidia GPU with 4GiB of VRAM. It's not very efficient but it can do these tasks a whole lot faster than the CPU, but the 4GiB limitation pretty much limits its use to only the tiniest models.

If this model can run inside of the 4GiB limit, that makes this infinitely more useful than existing models for me.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#192

Post by aalam »

jermaustin1 wrote: post #190
hmokiguess wrote: post #180

Got it to run on iPhone but was surprised to see they have some form of censorship and moderation on the input side on their client app. I thought a big part of local/offline AI was sovereignty, unfiltered, and censorship/bias resistance.

It looks like the text encoder is a separate bonsai model, so it can probably be obliterated or whatever it is called.

Abliteration. The word and its descriptions read like pure sci-fi

https://huggingface.co/blog/mlabonne/abliteration

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#194

Post by kevin_thibedeau »

stavros wrote: post #82
kevin_thibedeau wrote: post #71

Earlier quoted context omitted.

> so forget it. Which explains why you're using a dumb terminal to access compute services?

Basically, yes. We are on a website, after all.

I have a multi-core supercomputer to use locally as needed.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#195

Post by MrDrMcCoy »

mft_ wrote: post #32

Genuine question: is this solving a real problem? IME, the bottleneck when using diffusion models isn't storage space or memory, it's generation time. Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. I also note that these models are marginally slower than the small FLUX.2 model they're based on. Oka…

Lower memory use == higher speed. Memory bandwidth is conserved with less to transfer; this is the biggest bottleneck. Compressing your filesystem generally makes storage faster as well.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#196

Post by dogleash »

paulluuk wrote: post #135
flashman wrote: post #97

Twenty years ago, I don't think any of us were excited about a future internet where we couldn't trust whether what we were seeing or reading was genuine. I hope one day we'll be able to look back on this era as an aberration, like that scene in Mad Men where the Drapers fling their picnic rubbish onto the grass and drive away.

Twenty years ago my teachers were telling me not to use Wikipedia because you can't trust anything on the internet. You should never date someone you met through an app or website because they are 100% murderers. "The internet is for porn". Things have a way of improving over time, and people always overestimate societal risks of new tech in the beginning.

>Twenty years ago my teachers were telling me not to use Wikipedia because you can't trust anything on the internet.

Still can't. "ChatGPT can make mistakes." People still trust it, doesn't mean they should. Wiki's not as bad of a tertiary source as it used to be, but it's still a tertiary source and you had a research assignment. Even official authoritative sources can be un/intentionally wrong.

> You should never date someone you met through an app or website because they are 100% murderers.

This remains sound advice that teachers should continue giving children. Even (or morso) now that online dating has been normalized in the meantime. Do I have to explain?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#197

Post by nostrebored »

rickdeckard wrote: post #136
nostrebored wrote: post #114

Earlier quoted context omitted.

I am pretty excited. The factuality of important events has been distorted for most of history. Moving to a low information trust society is something that I think will be positive.

Ignore the naysayers, they are just jealous. You got it totally tight, not everyone get's it like we do. We are facing alot of backlash for our beliefs these days. Listen, I'm hosting this Telegram channel for people like us, where we can exchange free information without media bias, share the real facts and plan coordinated activities against these poisoning mainstream scumbags. I also have a 20% coupon code for Wam…

But this is exactly my point -- in a low information trust society things like this our noise. They work in high information trust societies because who would sell medical products that don't work? A doctor is leading Wamp®, surely a doctor would never lie to us for money?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#198

Post by nostrebored »

pjc50 wrote: post #158
nostrebored wrote: post #114

Earlier quoted context omitted.

I am pretty excited. The factuality of important events has been distorted for most of history. Moving to a low information trust society is something that I think will be positive.

Low trust societies are poorer because everyone has to spend much more effort on verifying everything. People give up business opportunities because they can't trust their partners. It becomes more nepotistic because people trust family over strangers. Low information trust societies get destroyed by pandemics of both physical viruses (due to anti-vax and medical distrust; we can see this happening again with Ebola)…

I've lived in low information trust societies, and they were much intellectually healthier than high trust societies, at least in the white collar communities I was in.

Moving there, I was shocked at how "conspiratorial" everybody seemed about everything. Living there, I was shocked out how often they were right. But it didn't impact people's likelihood to do things. I think they are actually orthogonal in a way that is unintuitive.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#199

Post by nostrebored »

ky-hy wrote: post #134
mungoman2 wrote: post #119

Earlier quoted context omitted.

Curious about this take, how do you mean? I understand the point of distorted facts, but what I’m not sure how things are improved by basically having no trust in any facts?

One of my professors at Uni a year ago was arguing that due to genAI we would have a shift of trust into established institutions/people, so government (I'm not American so I don't know if this is possible after many recent scandals for example), Universities, people with authority/knowledge, close family members that are trust worthy, knowledgeable and/or work in before mentioned institutions. So basically we would…

I've lived in a low information trust society and this was not the outcome at all. People trusted their local cliques much more, and there were local minima (e.g. a mainstreamish political party with leaders that are actively and dangerously anti semitic), but in general people were way more willing to engage with ideas.

One implication was people were more social and talked about ideas more. Thought had not been outsourced to arbiters in the way that it was in the U.S. People with authority, knowledge, and close family members were definitely inputs into what people thought, but by-and-large people still came to their own conclusions.

You got to see the gradient of thought that people actually had about issues. People would say their insane ideas out loud. You could disagree with people and have them actually engage with your perception of reality.

It was strictly better in my opinion.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#200

Post by normalaccess »

vineyardmike wrote: post #101
mirekrusin wrote: post #56

Earlier quoted context omitted.

It starts to be interesting when latency is better than average website.

I’m not sure if this is what you meant, but at 17k t/s, you start to compete with the speed of network calls. You could approach the point of generating an HTML/js/css page faster than some websites can be returned over the network.

When that happens, they will be able to rewrite reality in real time.
Post reply on HN