Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

snap-research.github.io

Post reply on HN
Print view View original

21–30 of 54 posts

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#21

Post by phire »

anotheryou wrote: post #15

Why focus on mobile? Near real-time rendering while entering a prompt on desktop would be even more amazing! Imagine e.g. UI sliders for adding weight to multi-prompts like "night time" (ideally sliding from [day time]:1, [night time]:0 to both zero, to all night with 0,1).

- It's significantly easier to monetize mobile apps (both with ads and in-app purchases)

- Open projects already dominate the desktop space

- There are a growing number of younger people who aren't really computer literate, or otherwise just use their phone as their primary computing device.

- Phones go into social situations where desktops/laptops don't.

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#23

Post by suyash »

schappim wrote: post #18

Sub 2 second generations on cell phones, nice! Better FID and CLIP scores than Stable Diffusion v1.5 with 50 steps, great! So are they gonna release the code, or do they only open-source ad-SDKs[1]? [1] https://github.com/orgs/Snapchat/repositories

I'll believe thier hypothesis when I can run the open source code on my iPhone in 2 seconds, doubt it's that fast.

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#24

Post by tudorw »

anotheryou wrote: post #15

Why focus on mobile? Near real-time rendering while entering a prompt on desktop would be even more amazing! Imagine e.g. UI sliders for adding weight to multi-prompts like "night time" (ideally sliding from [day time]:1, [night time]:0 to both zero, to all night with 0,1).

I'd like to be able to define camera movement as a a 3d path along around a geodesic in a multidimensional topological manifold :)

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#25

Post by jonplackett »

noduerme wrote: post #20
schappim wrote: post #19

Earlier quoted context omitted.

> Why focus on mobile? Snap Inc.

Oh you're so right. The face filter industrial complex is going to eat this up. And after them, plastic surgeons... God help us all.

Face Filter Industrial Complex is my new favourite term.

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#26

Post by asutekku »

suyash wrote: post #23
schappim wrote: post #18

Sub 2 second generations on cell phones, nice! Better FID and CLIP scores than Stable Diffusion v1.5 with 50 steps, great! So are they gonna release the code, or do they only open-source ad-SDKs[1]? [1] https://github.com/orgs/Snapchat/repositories

I'll believe thier hypothesis when I can run the open source code on my iPhone in 2 seconds, doubt it's that fast.

They included a video: https://www.youtube.com/watch?v=zK5PQ3Oj_L8

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#27

Post by prepend »

asutekku wrote: post #26
suyash wrote: post #23

Earlier quoted context omitted.

I'll believe thier hypothesis when I can run the open source code on my iPhone in 2 seconds, doubt it's that fast.

They included a video: https://www.youtube.com/watch?v=zK5PQ3Oj_L8

That demo doesn’t really do anything for me to know if it’s on device or not.

Without source code it could just be calling the OpenAI api.

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#28

Post by lelandfe »

prepend wrote: post #27
asutekku wrote: post #26

Earlier quoted context omitted.

They included a video: https://www.youtube.com/watch?v=zK5PQ3Oj_L8

That demo doesn’t really do anything for me to know if it’s on device or not. Without source code it could just be calling the OpenAI api.

The phone is in airplane mode

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#29

Post by prepend »

lelandfe wrote: post #28
prepend wrote: post #27

Earlier quoted context omitted.

That demo doesn’t really do anything for me to know if it’s on device or not. Without source code it could just be calling the OpenAI api.

The phone is in airplane mode

The issue is that it’s a controlled video from the author. I can still get Wi-Fi and Bluetooth in airplane mode so the airplane mode sign isn’t “proof” enough for me to accept the paper conclusions.

I hope it’s real, but posting a YouTube to assuage the “show me the implementation” isn’t going to help with my cynicism.

Re: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices Within Two Seconds

#30

Post by TeMPOraL »

devadvance wrote: post #4

From the paper: > In this work, we present the first text-to-image diffusion model that generates an image on mobile devices in less than 2 seconds. To achieve this, we mainly focus on improving the slow inference speed of the UNet and reducing the number of necessary denoising steps. As a layman, it's impressive and surprising that there's so much room for optimization here, given the number of hands on folks in the…

> As a layman, it's impressive and surprising that there's so much room for optimization here, given the number of hands on folks in the OSS space.

There's only so many folks in OSS space that are capable of doing work from this angle. There are more who could be micro-optimizing code, but the most end up developing GUIs and app prototypes and ad-hoc Python scripts that use the models.

At the same time, the whole field moves at ridiculously fast pace. There's room for optimization because the new model generations are released pretty much as fast as they're developed and trained, without stopping to tune or optimize them.

Also, there must be room for optimization given how ridiculously compute-expensive training and inference still is. Part of my intuition here is that current models do roughly similar things to what our brains do, and brains manage to do these things fast with some 20-50 watts. Sure, there are a lot of differences between NN models and biological brains, but to a first approximation, this is a good lower bound.

Post reply on HN