Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

creator.nightcafe.studio

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#11

Post by nonbirithm »

protogenes wrote: post #3

Cool! There are at least 2 more VQGAN + CLIP systems: - makeai.art (also see https://news.ycombinator.com/item?id=28070149 ) - https://NeuralBlender.com

These are cool. I wish it was possible to move beyond Colab for generative type ML as the UX is horrible and there is no practical way to use them on mobile. Unfortunately it seems to be a problem of who has the hardware and who doesn't. Hopefully as time passes it will be easier to host these kinds of services, and not have to feel hopeless after the next AI Dungeon-type corporate implosion.

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#13

Post by GusRuss89 »

nonbirithm wrote: post #11
protogenes wrote: post #3

Cool! There are at least 2 more VQGAN + CLIP systems: - makeai.art (also see https://news.ycombinator.com/item?id=28070149 ) - https://NeuralBlender.com

These are cool. I wish it was possible to move beyond Colab for generative type ML as the UX is horrible and there is no practical way to use them on mobile. Unfortunately it seems to be a problem of who has the hardware and who doesn't. Hopefully as time passes it will be easier to host these kinds of services, and not have to feel hopeless after the next AI Dungeon-type corporate implosion.

Yep, NightCafe has struggled for years with the expense of running algorithms on GPUs. It started out free, but as soon as you start to get any kind of volume, costs balloon.

I almost gave up on NightCafe when I was trying to make money from selling prints to people. Then, one user generated 3,000 creations in a week, which cost him nothing and me $300. I decided to spend the weekend implementing a credit system to prevent abuse, and was surprised to find that more people were willing to pay for credits than I expected.

It's still not much and I've had to spend a lot of effort on reducing costs in various ways just to be able to break even. But I love working on this app and have high hopes that _one day_ it will become profitable enough for me to work on it full time.

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#14

Post by mattway »

The text to image results look pretty awesome, but I feel like they all have a "neural net" quality to them. Whilst the style transer results come out more unique. Do you think it would become possible to combine the two techniques? Eg. Generate me a pic of this text, but in this images style?

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#16

Post by GusRuss89 »

mattway wrote: post #14

The text to image results look pretty awesome, but I feel like they all have a "neural net" quality to them. Whilst the style transer results come out more unique. Do you think it would become possible to combine the two techniques? Eg. Generate me a pic of this text, but in this images style?

The Colab notebooks actually have a "target image" parameter. I haven't added it to NightCafe Creator yet, and haven't even experimented much with it on Colab, but it's definitely on my to-do list.

However, adding keywords like "watercolour painting", "van gogh painting" etc goes a long way to getting more precise results.

One big difference between style transfer and GAN art is that style transfer never changes the shape of things. E.g. you can't put in a woman's face and a cubist painting and get out a rearranged version of the face (as cubists do). It will have all the hallmarks of cubism, but it's still recognisable as the original photo.

With GAN art though, you can put that photo in as a start image, say "Cubist painting of a woman's face", and the GAN will actually rearrange the face like a cubist would.

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#17

Post by mkishi »

I've been meaning to experiment with GAN art for a while now, but never managed to actually study or run any notebooks. I saw NightCafe Creator mentioned in another thread earlier today, and it made it too easy not to try a few prompts. Very neat first experience!

https://creator.nightcafe.studio/creation/EqWtT6LUB3WwVd3T56...

https://creator.nightcafe.studio/creation/iH0oqPjpo4oEHvxwlB...

Does it support the pipes and weights I saw mentioned in a few notebooks, eg. "motorbike:75 | bicycle:25 | car:0"?

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#18

Post by GusRuss89 »

mkishi wrote: post #17

I've been meaning to experiment with GAN art for a while now, but never managed to actually study or run any notebooks. I saw NightCafe Creator mentioned in another thread earlier today, and it made it too easy not to try a few prompts. Very neat first experience! https://creator.nightcafe.studio/creation/EqWtT6LUB3WwVd3T56... https://creator.nightcafe.studio/creation/iH0oqPjpo4oEHvxwlB... Does it support the pipes a…

It doesn't support that yet, but it will soon.

I will build an interface for it rather than a prompt syntax. My target audience is a bit less technical than the average Google Colab user.

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#19

Post by math0ne »

Really cool, if there are just two terms is is basically two images combined?

existential dread simulator: https://nightcafe-studio.imgix.net/jobs/GFM872ChxPF4Xeb2jTiC...

graffiti sunset: https://nightcafe-studio.imgix.net/jobs/MJfXTM5rFdNqHcR5xBRG...

Re: Show HN: NightCafe Creator – Create art from a text prompt using VQGAN+CLIP

#20

Post by GusRuss89 »

math0ne wrote: post #19

Really cool, if there are just two terms is is basically two images combined? existential dread simulator: https://nightcafe-studio.imgix.net/jobs/GFM872ChxPF4Xeb2jTiC... graffiti sunset: https://nightcafe-studio.imgix.net/jobs/MJfXTM5rFdNqHcR5xBRG...

It's not, but I can see why the "Graffiti sunset" would make you think that. It does seem to be two different images combined.

I won't claim to have a good knowledge of the inner workings, but basically CLIP will give a score for how well the image matches your prompt, and that's used as a loss function for the GAN.

On the Colab notebooks you can specify multiple prompts with different weights, in which case I assume it has multiple loss functions that it's trying to optimise for.

I'm more of a web app guy than a deep tech guy. Some of what I wrote can probably be corrected by someone with a better knowledge of the ML.

Post reply on HN