Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: New AI edits images based on text instructions

github.com

Post reply on HN
Print view View original

81–90 of 241 posts

Re: Show HN: New AI edits images based on text instructions

#81

Post by perfopt »

bryced wrote: post #77
perfopt wrote: post #75

Earlier quoted context omitted.

I keep seeing this even when the prompt is unchanged Downloading https://huggingface.co/runwayml/stable-diffusion-v1-5/resolv... from huggingface Loading model /home/hrishi/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/889b629140e71758e1e0006e355c331a5744b4bf/v1-5-pruned-emaonly.ckpt onto cuda backend... followed by a download

That is strange. I'm not sure what would cause that unless it was running in some ephemeral environment. What OS? Can you open a github issue with a screenshot?

Ubuntu 22.04.1 LTS. I'll file a issue

Re: Show HN: New AI edits images based on text instructions

#82

Post by bryced »

do_anh_tu wrote: post #27
nullish_signal wrote: post #3

>11GB VRAM Aaarrrgghh let me know when it's down to 4GB like Stable Diffusion The prompt-based masking sounds incredible, with either pixel +/- or Prompt Relevance +/- VERY impressive img2img capabilities!

Or, if there is a Colab version, I’d happy to pay Google for premium GPU.

It does work in non-pro colab apparently. Here you go: https://colab.research.google.com/drive/1rOvQNs0Cmn_yU1bKWjC...

Re: Show HN: New AI edits images based on text instructions

#83

Post by userbinator »

bryced wrote: post #76
nicbou wrote: post #41

Can it make it pop? Because that was the #1 request I remember dealing with.

I tried it out :-) `aimg edit assets/girl_with_a_pearl_earring.jpg "make it pop" --prompt-strength 40 --gif` https://user-images.githubusercontent.com/1217531/213912442-...

I was expecting something balloon-like to appear, and was not disappointed.

Re: Show HN: New AI edits images based on text instructions

#85

Post by Der_Einzige »

c7b wrote: post #40
chadcmulligan wrote: post #29

Earlier quoted context omitted.

Haven't we been here before? - see self driving cars.

LLMs and Image AIs are the opposite of self-driving cars. "Everybody" had concrete expectations for at least half a decade now that the moment where self-driving cars would surpass human ability was imminent, yet the tech hasn't lived up to it (yet). While practically nobody was expecting AI to be able to do the jobs of artists, programmers or poets anywhere near human level anytime soon, yet here we are.

Still bad at poetry due to the tokenizer though. I wrote a whole paper on how to fix it: https://paperswithcode.com/paper/most-language-models-can-be...

Re: Show HN: New AI edits images based on text instructions

#86

Post by bryced »

TekMol wrote: post #8

How can I try this? Can this be run on a Digitalocean VM? I looked around on DO's products, but none seems to advertise that it has a GPU. So maybe it is not possible?

Here is a google colab you can try it in: https://colab.research.google.com/drive/1rOvQNs0Cmn_yU1bKWjC...

Re: Show HN: New AI edits images based on text instructions

#87

Post by ColonelPhantom »

GordonS wrote: post #63

What are the most affordable GPUs that will run this? (it said it needs CUDA, min 11GB VRAM, so I guess my relatively puny 4GB 570RX isn't going to cut it!)

The cheapest NVidia GPU with 11+GB VRAM is probably the 2060 12GB, although the 3060 12GB would be a better choice.

The setup.py file seems to indicate that PyTorch is used, which I think can also run on AMD GPUs, provided you are on Linux.

Re: Show HN: New AI edits images based on text instructions

#88

Post by rjh29 »

c7b wrote: post #45
xdfgh1112 wrote: post #37

Earlier quoted context omitted.

I think that "prompt engineering" stuff went away when ChatGPT came out.

Has it? I mean, maybe the idea of people doing this as a long-time career has, but practically, I still find it a challenge to get those AIs to do exactly what I want. I've played around with Dreambooth-style extensions now, and that goes some way for some applications, and I'm excited to try OP's solution, but in my experience, it is still a bit of a limitation for working with those AIs right now .

Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.

Re: Show HN: New AI edits images based on text instructions

#89

Post by TeMPOraL »

> Here are some examples of transformations it can make: Golden gate bridge:

I'm on mobile so can't try this myself now. Can it add a Klingon bird of prey flying under the Golden Gate Bridge, and will "add a Klingon bird of prey flying under the Golden Gate Bridge" prompt/command be enough?

Re: Show HN: New AI edits images based on text instructions

#90

Post by c7b »

bryced wrote: post #69
c7b wrote: post #50

Many thanks to the OP, can't wait to try this out! I have a question I'm hoping to slide in here: I remember there were also solutions for doing things like "take this character and now make it do various things". Does anyone remember what the general term for that was, and some solutions (pretty sure I've seen this on here, apparently forgot to bookmark). PS: I'm not trying to make a comic book, I'm trying to help a…

dreambooth perhaps?

Dreambooth is what I'm using now, but I think I remember the concept had a specific name, something like 'context transfer' or so (pretty sure that was not the term) and tools that were pretty good at it that came out before Dreambooth. If I could at least remember the term it might be easier to search for them.

Dreambooth is ok at it, but it requires multiple images (you often read 30, but I've actually had decent results with as few as five) to recognize what it's supposed to replicate. I remember there were tools that were more adapted to the workflow "create a humanoid cartoon character with a bunny face", pick one image that you like, and then "now show that same character in scene X, eg teaching in a classroom" or "wearing a cowboy outfit".

Post reply on HN