Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1-Bit Bonsai Image 4B Image Generation for Local Devices

prismml.com

Post reply on HN
Print view View original

111–120 of 214 posts

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#111

Post by ToValueFunfetti »

lwansbrough wrote: post #104
IncreasePosts wrote: post #102

Earlier quoted context omitted.

Prisoner 1: so, what are you in for? Prisoner 2: I made a picture of a nice sunset over the ocean

If it were illegal it wouldn’t be readily available. You’d have to seek it out. People seeking it out wouldn’t be using it to generate a sunset.

If it were illegal to generate a sunset, people would absolutely seek it out to generate a sunset, if just as an act of civil disobedience. Look at how people reacted to being told sharing a number is illegal[1]. Why would they act differently about an illegal computation?

[1]https://en.wikipedia.org/wiki/AACS_encryption_key_controvers...

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#113

Post by Brisk4t »

barnas2 wrote: post #34
lumost wrote: post #10

I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than…

A company called Taalas is working on something like that. Not Opus4.6 quality, but I'm sure they're targeting larger models. Currently they're using a LLama 8B model. It runs at ~17k tokens per second, and you can test it at https://chatjimmy.ai/ .

I'm rooting for them HARD but they've been quiet since their last (and only) blog. X and LinkedIn are empty too. I really hope it wasn't a pipe dream.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#114

Post by nostrebored »

flashman wrote: post #97

Twenty years ago, I don't think any of us were excited about a future internet where we couldn't trust whether what we were seeing or reading was genuine. I hope one day we'll be able to look back on this era as an aberration, like that scene in Mad Men where the Drapers fling their picnic rubbish onto the grass and drive away.

I am pretty excited. The factuality of important events has been distorted for most of history. Moving to a low information trust society is something that I think will be positive.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#115

Post by AbraKdabra »

flashman wrote: post #97

Twenty years ago, I don't think any of us were excited about a future internet where we couldn't trust whether what we were seeing or reading was genuine. I hope one day we'll be able to look back on this era as an aberration, like that scene in Mad Men where the Drapers fling their picnic rubbish onto the grass and drive away.

Aberration? That seems like an extreme overreaction.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#116

Post by ppeetteerr »

mft_ wrote: post #32

Genuine question: is this solving a real problem? IME, the bottleneck when using diffusion models isn't storage space or memory, it's generation time. Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. I also note that these models are marginally slower than the small FLUX.2 model they're based on. Oka…

Yes, size and performance are not only problems for local LLMs, they are problems for frontier LLM companies like OpenAI and Anthropic. The latter still lose a ton of money on inference and advances in efficient, performant models helps their bottom line.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#117

Post by cheesecakegood »

joblessjunkie wrote: post #67
mft_ wrote: post #32

Genuine question: is this solving a real problem? IME, the bottleneck when using diffusion models isn't storage space or memory, it's generation time. Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. I also note that these models are marginally slower than the small FLUX.2 model they're based on. Oka…

We are in an era of extreme demand for GPU and limited supply. Every inference we push to the edge frees cloud resources for other tasks. Every efficiency gain increases what we can achieve with existing resources. If images can be rendered with half as much compute, we need half as many GPUs.

… or generate twice as many images. Maybe not quite, but if we’ve seen anything with AI so far is that it fits Parkinson’s law pretty well.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#118

Post by lumost »

SiempreViernes wrote: post #63
lumost wrote: post #57

Earlier quoted context omitted.

"Design me a 3d printable rocket engine for a hobby rocket project. Verify it's design in a full simulation. Iterate until it works reliably in simulation based on a verified printable design on a consumer laser sintering device (or substitute contract manufacture for under 1000 dollars)." This is a hobby version of a project, but you can imagine commercial versions of the same prompt for new databases, genomics stud…

From the prompt it seems evident the envisioned user doesn't have an interest in designing the motor themselves, so why not simply buy a stock motor?

I can't put a 10 page narrative on how my specific motor should work into a hacker news post ;) you can also imagine the above where the goal is to have the ai exceed the performance of stock motors.

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#119

Post by mungoman2 »

nostrebored wrote: post #114
flashman wrote: post #97

Twenty years ago, I don't think any of us were excited about a future internet where we couldn't trust whether what we were seeing or reading was genuine. I hope one day we'll be able to look back on this era as an aberration, like that scene in Mad Men where the Drapers fling their picnic rubbish onto the grass and drive away.

I am pretty excited. The factuality of important events has been distorted for most of history. Moving to a low information trust society is something that I think will be positive.

Curious about this take, how do you mean?

I understand the point of distorted facts, but what I’m not sure how things are improved by basically having no trust in any facts?

Re: 1-Bit Bonsai Image 4B Image Generation for Local Devices

#120

Post by fulafel »

mft_ wrote: post #32

Genuine question: is this solving a real problem? IME, the bottleneck when using diffusion models isn't storage space or memory, it's generation time. Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway. I also note that these models are marginally slower than the small FLUX.2 model they're based on. Oka…

> Lots of models will run on 8-12 GB 1080-generation GPUs onwards, or on Macs with similar memory, which are probably the bottom end from a GPU power perspective anyway.

Not the bottom end - most people are on laptops or mobile devices that are much lower GPU power than this.

Post reply on HN