Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MobileDiffusion: Rapid text-to-image generation on-device

blog.research.google

Post reply on HN
Print view View original

11–20 of 67 posts

Re: MobileDiffusion: Rapid text-to-image generation on-device

#11

Post by tussa »

cuuupid wrote: post #6

Google has fallen so far. Both Inception and Mobilenet were released openly and changed the entire AI world. Nowadays we just get blog posts about results that were supposedly achieved, an accompanying paper that can’t be reproduced (because of Google’s magical “private datasets”), and some screencaps of a cool application of the tech that is virtually guaranteed to never make it to product.

Probably because the actual product is garbage.

Remember the Google assistant demo, where it booked a table at a restaurant?

That never materialized. Google assistant is just eating crayons today.

Re: MobileDiffusion: Rapid text-to-image generation on-device

#13

Post by ollin »

some points that stood out to me:

1. they made a lot of careful tweaks to the unet network architecture - it seems like they ran many different ablations here ("In total, our endeavor consumes approximately 512 TPUs spanning 30 days").

2. the model distillation is based on previous UFOGen work from the same team https://arxiv.org/abs/2311.09257 (hence the UFO graphic in the diffusion-gan diagram)

3. they train their own 8-channel latent encoder / decoder ("VAE") from scratch (similar to Meta's Emu paper) instead of using the SD VAEs like many other papers do

4. they use an internal dataset of 150m image/text pairs (roughly the size of laion-highres)

5. they also reran SD training from scratch on this dataset to get their baseline performance

Re: MobileDiffusion: Rapid text-to-image generation on-device

#14

Post by geor9e »

SushiHippie wrote: post #4

Kind of funny that they show the iphone 15 pro and the Samsung S24 in the comparison chart, but not their own phone the google pixel 8. (I know it will perform worse than both phones)

Pixel marketing touts the NPU chip, which sounds ideal for accelerating a model like this. They might have reasons for leaving it out. Perhaps it's planned to ship with a new model - announcing a new phone in a model paper would be weird. I remember them advertising some Pixel-only photo editing features in the past.

Re: MobileDiffusion: Rapid text-to-image generation on-device

#15

Post by refulgentis »

geor9e wrote: post #14
SushiHippie wrote: post #4

Kind of funny that they show the iphone 15 pro and the Samsung S24 in the comparison chart, but not their own phone the google pixel 8. (I know it will perform worse than both phones)

Pixel marketing touts the NPU chip, which sounds ideal for accelerating a model like this. They might have reasons for leaving it out. Perhaps it's planned to ship with a new model - announcing a new phone in a model paper would be weird. I remember them advertising some Pixel-only photo editing features in the past.

It's great in theory until you have to pierce through several organizational boundaries to actually leverage it. Too sclerotic for it to happen regularly at scale.

Re: MobileDiffusion: Rapid text-to-image generation on-device

#16

Post by refulgentis »

tussa wrote: post #11
cuuupid wrote: post #6

Google has fallen so far. Both Inception and Mobilenet were released openly and changed the entire AI world. Nowadays we just get blog posts about results that were supposedly achieved, an accompanying paper that can’t be reproduced (because of Google’s magical “private datasets”), and some screencaps of a cool application of the tech that is virtually guaranteed to never make it to product.

Probably because the actual product is garbage. Remember the Google assistant demo, where it booked a table at a restaurant? That never materialized. Google assistant is just eating crayons today.

Yeah it did

But to your point its not as impressive as it was sold: glorified reverse phone tree. But it is what it was sold as, and it is available.

Re: MobileDiffusion: Rapid text-to-image generation on-device

#17

Post by causality0 »

geor9e wrote: post #14
SushiHippie wrote: post #4

Kind of funny that they show the iphone 15 pro and the Samsung S24 in the comparison chart, but not their own phone the google pixel 8. (I know it will perform worse than both phones)

Pixel marketing touts the NPU chip, which sounds ideal for accelerating a model like this. They might have reasons for leaving it out. Perhaps it's planned to ship with a new model - announcing a new phone in a model paper would be weird. I remember them advertising some Pixel-only photo editing features in the past.

All the Pixel AI stuff runs on the cloud anyway. Just try using it in airplane mode.

Re: MobileDiffusion: Rapid text-to-image generation on-device

#18

Post by dvngnt_ »

causality0 wrote: post #17
geor9e wrote: post #14

Earlier quoted context omitted.

Pixel marketing touts the NPU chip, which sounds ideal for accelerating a model like this. They might have reasons for leaving it out. Perhaps it's planned to ship with a new model - announcing a new phone in a model paper would be weird. I remember them advertising some Pixel-only photo editing features in the past.

All the Pixel AI stuff runs on the cloud anyway. Just try using it in airplane mode.

not entirely true. translation is done on device

Re: MobileDiffusion: Rapid text-to-image generation on-device

#19

Post by inquirerGeneral »

tussa wrote: post #11
cuuupid wrote: post #6

Google has fallen so far. Both Inception and Mobilenet were released openly and changed the entire AI world. Nowadays we just get blog posts about results that were supposedly achieved, an accompanying paper that can’t be reproduced (because of Google’s magical “private datasets”), and some screencaps of a cool application of the tech that is virtually guaranteed to never make it to product.

Probably because the actual product is garbage. Remember the Google assistant demo, where it booked a table at a restaurant? That never materialized. Google assistant is just eating crayons today.

It did work, I used it myself. A quick search shows others who had my experience. This was late 2019 for me. Here's the first link and the Google post on rolling out in summer 2019:

"Google Assistant just made a dinner reservation for me... I knew this was coming... but mind blown!" https://www.reddit.com/r/googlehome/comments/ezv3us/google_a...

"Now, you can use it on all Pixel phones in 43 U.S. states.

All it takes is a few seconds to tell your Assistant where you'd like to go. Just ask the Assistant on your phone, “Book a table for four people at [restaurant name] tomorrow night.” The Assistant will then call the restaurant to see if it can accommodate your request. Once your reservation is successfully made, you’ll receive a notification on your phone, an email update and a calendar invite so you don’t forget."

https://blog.google/products/assistant/book-table-google-ass...

Re: MobileDiffusion: Rapid text-to-image generation on-device

#20

Post by adamzochowski »

dvngnt_ wrote: post #18
causality0 wrote: post #17

Earlier quoted context omitted.

All the Pixel AI stuff runs on the cloud anyway. Just try using it in airplane mode.

not entirely true. translation is done on device

Not entirely true either. If it thinks it has network but it's flakey, it won't translate offline, it will say there is network error and will give you a button to retry. No button to do offline.

Additionally, in airplane mode it heavily doesn't want to translate, in my use case I have to go to saved translations as otherwise it won't even let me type what I need to translate.

Post reply on HN