[deleted]
Show HN: New AI edits images based on text instructions
91–100 of 241 posts
Re: Show HN: New AI edits images based on text instructions
#92The language of high-level art-direction can be way more complex than one might assume. I wonder how this model might cope with the following:
‘Decrease high-frequency features of background.’
‘Increase intra-contrast of middle ground to foreground.’
‘Increase global saturation contrast.’
‘Increase hue spread of greens.’
Re: Show HN: New AI edits images based on text instructions
#93Earlier quoted context omitted.
Has it? I mean, maybe the idea of people doing this as a long-time career has, but practically, I still find it a challenge to get those AIs to do exactly what I want. I've played around with Dreambooth-style extensions now, and that goes some way for some applications, and I'm excited to try OP's solution, but in my experience, it is still a bit of a limitation for working with those AIs right now .
Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.
Absolutely. I briefly thought about asking ChatGPT to write a prompt, but then I remembered that the training corpus is probably older than those tools (I heard that if you ask it the right way, it will tell you that its corpus ended in 21 - whether it's true or not, it sounds plausible). But that's a truly temporary issue, the respective subreddits probably have enough information to train an AI for prompt engineering already (if you start from a strong foundation like the latest GPT versions).
Plus, who knows whether future models won't be able to integrate those different modes much better (along those lines https://www.deepmind.com/publications/a-generalist-agent).
Re: Show HN: New AI edits images based on text instructions
#94Earlier quoted context omitted.
Has it? I mean, maybe the idea of people doing this as a long-time career has, but practically, I still find it a challenge to get those AIs to do exactly what I want. I've played around with Dreambooth-style extensions now, and that goes some way for some applications, and I'm excited to try OP's solution, but in my experience, it is still a bit of a limitation for working with those AIs right now .
Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.
I don't get it. Pre-ChatGPT prompt engineering was a BS exercise in guessing how a given model's front-end tokekizes and processes the prompt. ChatGPT made it only more BS. But I've seen a paper the other day, implementing more structured, formal prompt language, with working logic operators implemented one layer below - instead of adding more cleverly structured English, they were stepping the language model with variations of the prompt (as determined by the operators), and did math on probability distributions of next tokens the model returned. That, to me, sounds like valid, non-BS approach, and strictly better than doubling down on natural language.
Re: Show HN: New AI edits images based on text instructions
#95What are the most affordable GPUs that will run this? (it said it needs CUDA, min 11GB VRAM, so I guess my relatively puny 4GB 570RX isn't going to cut it!)
It works fine on CPUs. Takes about a minute to generate images on my 8 core i7 desktop.
Re: Show HN: New AI edits images based on text instructions
#96Earlier quoted context omitted.
LLMs and Image AIs are the opposite of self-driving cars. "Everybody" had concrete expectations for at least half a decade now that the moment where self-driving cars would surpass human ability was imminent, yet the tech hasn't lived up to it (yet). While practically nobody was expecting AI to be able to do the jobs of artists, programmers or poets anywhere near human level anytime soon, yet here we are.
Still bad at poetry due to the tokenizer though. I wrote a whole paper on how to fix it: https://paperswithcode.com/paper/most-language-models-can-be...
Great work, congratulations! One question, if I understood it right you based your demo on GPT-2 - what is your experience working with those open-source language AIs. In terms of computational requirements and performance?
I'm really fascinated by all the tools the OS community is building based on StableDiffusion (like OP's), which compares favourably with the latest closed-source models like Dall-E or Midjourney, and can run reasonably well on a high-end home computer or a very reasonably-sized cloud instance. For language models, it seems the requirements are substantially higher, and it's hard to match the latest GPT versions in terms of quality.
Re: Show HN: New AI edits images based on text instructions
#97What are the most affordable GPUs that will run this? (it said it needs CUDA, min 11GB VRAM, so I guess my relatively puny 4GB 570RX isn't going to cut it!)
I'm running on a 2080 TI and an edit runs in 2 seconds. On my Apple M1 Max 32Gb edits take about 60 seconds.
If this was all packaged into a desktop app (e.g. Tauri or electron) how big would the app be? I'd imagine you could get it down to < 500MB (even if you packaged miniconda with it).
Re: Show HN: New AI edits images based on text instructions
#98Super nice. Would this work if I have my own version of fine-tuned SD? Also, curious how / whether this is different from img2img released by SD. Thanks!
Re: Show HN: New AI edits images based on text instructions
#99Earlier quoted context omitted.
Oh yeah it's definitely still an issue right now! But I think the power of ChatGPT's ability to understand and execute instructions has convinced most people that "prompt engineering" isn't going to be a career path in the future.
I don't get it. Pre-ChatGPT prompt engineering was a BS exercise in guessing how a given model's front-end tokekizes and processes the prompt. ChatGPT made it only more BS. But I've seen a paper the other day, implementing more structured, formal prompt language, with working logic operators implemented one layer below - instead of adding more cleverly structured English, they were stepping the language model with va…
Think about the problem in an end-to-end fashion: the user has an idea of what sort of image they want, they just need an interface to tell the machine. A combination of natural language plus optional image/video input is probably the most intuitive interface we can provide (at least until we've made far more progress on reading brain signals more directly).
How exactly we get there, by adding layers like on top like language models, or adding layers below like what you described, doesn't seem like such a fundamental difference. It's engineering, you try different approaches, vary your parameters and see what works best. And from the onset, natural language does seem like a good candidate for encoding nuances like "make it pink, but not cheesy" or "has the vibes of a 50's Soviet propaganda poster, but with friendlier colors".
Re: Show HN: New AI edits images based on text instructions
#100What are the most affordable GPUs that will run this? (it said it needs CUDA, min 11GB VRAM, so I guess my relatively puny 4GB 570RX isn't going to cut it!)
The cheapest NVidia GPU with 11+GB VRAM is probably the 2060 12GB, although the 3060 12GB would be a better choice. The setup.py file seems to indicate that PyTorch is used, which I think can also run on AMD GPUs, provided you are on Linux.
I really want these ML libraries to get smarter with use of VRAM. Just cos I don't have enough VRAM shouldn't stop me computing the answer. It should just transfer in the necessary data from system ram as needed. Sure, it'll be slower, but I'd prefer to wait 1 minute for my answer rather than getting an error.
And if I don't have enough system RAM, bring it in from a file on disk.
Tha majority of the ram is consumed by big weights matrices, so the framework knows exactly which bits of data are needed when and in what order, so should be able to do efficient streaming data transfers to have all the data in the right place at the right time. It would be far more efficient than 'swap files' that don't know ahead of time what data will be needed so impact performance severely.