I know 29 minutes is long but theoretically you can have all the images you ever want in a small 6gb package and run inference on (nearly) everything. That's fucking amazing.
OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
21–30 of 88 posts
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#22So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?
I mean, diffusion models tend to be less computationally expensive than say CNNs or LLMs, so probably? And before than ppl ran SVMs, random forest, and other forms of non-gpu intensive ML algorithms locally as well...
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#23It reminds me of the time it took to generate mandelbrots on home computers in the 80s.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#24Earlier quoted context omitted.
Every Google Home device is already running an ML model to do speech recognition to recognize the "hey Google" wake word, so sooner than 10 years. The Raspberry Pi Zero is a particularly underpowered device for this. Doing it on the Coral TPU accelerator plugged into a pi zero would take less than 30 mins. Doing it on an iPhone 15 would take less time. Doing it on a Pixel 8 would be faster. Not to diminish getting it…
There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.
Siri came out 12 years ago, wake words are probably a bad example.
Better examples are the magic eraser on my 2 generation old pixel phone or the fact llama2 runs genuinely fast on a Mac mini
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#25So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?
Probably less than 5 years, maybe 2
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#26So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?
The bottleneck is probably the availability of lithography machines that can make ubiquitous chips for processing that much data quickly enough without heating or drawing electricity too much.
Not too far in the future every device will have some 5nm or better tech LLM chip inside and devices understanding natural language will be the norm.
By dumb machines, people will mean machines that have to be programmed by people using ancient techniques where everything the machine is supposed to do is written step by step in a low level computer language like JavaScript.
Nerds will be making demos of doing something incredibly fast by directly writing the algorithms by hand, and will be annoyed by the fact that something that can be done in 20 lines of code on few hundred MB of RAM in NodeJS now requires a terabyte of RAM.
Dumb phone will be something like iPhone 15 pro or Pixel 8 Pro where you have separate apps for each thing you do and you can't simply ask the device to do it for you.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#27Submitted title was "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes", which is good to know in order to understand some of the comments posted before I changed the title.
Submitters: if you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#28Earlier quoted context omitted.
But that's not the point, obviously. Sometimes, being slow is a feature. Besides, a 4090 costs more than a small car.
$1600 is more than a car? I feel like you can't even find driveable cars that will last 100 miles at that price point anymore.
You probably can, but it'll take some time. The supply of reasonable reliable $500-$1000 beaters is a lot less than it used to be.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#29Earlier quoted context omitted.
There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.
Siri came out 12 years ago, wake words are probably a bad example. Better examples are the magic eraser on my 2 generation old pixel phone or the fact llama2 runs genuinely fast on a Mac mini
Siri’s wake word stuff is also terrible, she gets constantly activated whenever I have my Apple Watch near running water, frying food or anything else that makes a white noise-type sound.
Re: OnnxStream: Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2
#30Submitted title was "Stable Diffusion Turbo on a Raspberry Pi Zero 2 generates an image in 29 minutes", which is good to know in order to understand some of the comments posted before I changed the title. Submitters: if you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn…
Wait, but the new title doesn't seem to be correct