Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a Stable Diffusion prompt changes its output for the style of 1500 artists

gorgeous.adityashankar.xyz

Post reply on HN
Print view View original

191–200 of 205 posts

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#191

Post by Bakary »

Daub wrote: post #165

I lecture in painting, and to me some of these are truly impressive. Not surprisingly, those artists is not predominantly painting (e.g. Josph Beuys) or is predominantly linear (Audrey Beardly) do not fare so well. There is a lot of talk in our department as to how we might prepare our students for this technology. It is scary how fast it is growing, and how it is spreading to things like 3D and texturing. One of my…

I'd be interested in hearing your take because you are actively involved in this field. From my perspective outside of it, it seems that it's going to be an absolute bloodbath in terms of opportunities for people to actually live as artists (excluding those who work in mediums that can't be represented on a 2D screen).

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#192

Post by intelVISA »

odessacubbage wrote: post #145
intelVISA wrote: post #128

Earlier quoted context omitted.

As with programming, an AI model cannot replace the key parts but can help automate the monotony. For me it's exciting to use as placeholder art and then have a 'real' artist review it.

i find myself mentally unable to comprehend people who believe that the drawing part of drawing is monotony. discovering that this mindset not only exists but is widespread has been equally as disturbing as any ai advancement.

Not me, I enjoy drawing; perhaps I should've used the word 'manual' instead

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#193

Post by bawolff »

otabdeveloper4 wrote: post #180
krapp wrote: post #115

Earlier quoted context omitted.

Whatever else this emergent "creative AI" phenomenon may or may not do, it's definitely touching nerves in people who still believe there's something ineffable and transcendent about the creative experience.

Stable Diffusion is literally copy-pasting existing artwork. It's not creating anything new. If anything, this sort of "AI" only makes human ineffable creative experiences even more valuable, because without it there would be no training set and no "AI".

Whether or not its creating something "new" is debatable, but its clearly not copy pasting. It is basing the artwork on a combination of other sources, nothing is created de novo, but that is very different than copy & pasting

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#194

Post by bawolff »

jhbadger wrote: post #187
bawolff wrote: post #95

Earlier quoted context omitted.

Its kind of a weird complaint. If i am having a conversation with someone, i wouldn't be concerned about knowing the set of all possible nouns.

SD isn't a person you can converse with. It's just a program trained on captions and can do no more than what's in them. It's like those old adventure games that would always complain "I don't that word" except even worse because SD will happily make a picture with words it doesn't know and not tell you.

I think anyone who has both played an IF game and has played with stable diffusion knows there is a world of difference between the two.

The main difference is that coming up with a word SD doesn't know that's not contrived is really difficult. In an IF game, you are constantly guessing the correct word.

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#195

Post by EVa5I7bHFq9mnYK »

chaxor wrote: post #139
EVa5I7bHFq9mnYK wrote: post #93

Interesting, how with billions of nodes and supposed "intelligence", the network hasn't been able to deduce a simple concept of symmetry in human faces. All of the eyes and all of the lips in all of the pictures are asymmetrical, which easily gives AI generated images away.

Many of the artists in this list specifically try not* to have this symmetry present in the faces. It's what makes many styles of art separate from just taking a photo or simply going outside. The system used here is actually astoundingly good at producing many artists styles because it's not going for symmetry.

It's true not only for these artist sets, but for photo sets as well, first noticed that with ThisPersonDoesNotExist.

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#196

Post by spijdar »

nl wrote: post #177
spijdar wrote: post #41

Earlier quoted context omitted.

I don't think the underlying model is word based, but character based. You could download the caption data for LAION and grep that, but it's not strictly 1:1 with what SD was trained against.

No, it's word based. The vocabulary is here: https://huggingface.co/openai/clip-vit-base-patch32/resolve/... It is contextual though, so words in different orders mean different things.

Huh, interesting, I had just ... assumed CLIP's tokenizer was character based, like GPT's was. At least, I think GPT's is character based?

Is there any reason it couldn't be character based, besides the (presumably very large) increase in resources needed to train and run inference? This is all way out of my league, but seems like you could get interesting results from this, since (by my caveman understanding) this hypothetical transformer could make some sense of words it had never seen before, so spelling variants or neologisms and such.

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#197

Post by nl »

spijdar wrote: post #196
nl wrote: post #177

Earlier quoted context omitted.

No, it's word based. The vocabulary is here: https://huggingface.co/openai/clip-vit-base-patch32/resolve/... It is contextual though, so words in different orders mean different things.

Huh, interesting, I had just ... assumed CLIP's tokenizer was character based, like GPT's was. At least, I think GPT's is character based? Is there any reason it couldn't be character based, besides the (presumably very large) increase in resources needed to train and run inference? This is all way out of my league, but seems like you could get interesting results from this, since (by my caveman understanding) this h…

I started a proper reply but had to board a plane.

It's actually a byte-pair encoded (BPE is better than character encoding but can do the things you mentioned) list of things that includes words. You can find common English suffixes in it listed separately too.

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#198

Post by spijdar »

nl wrote: post #197
spijdar wrote: post #196

Earlier quoted context omitted.

Huh, interesting, I had just ... assumed CLIP's tokenizer was character based, like GPT's was. At least, I think GPT's is character based? Is there any reason it couldn't be character based, besides the (presumably very large) increase in resources needed to train and run inference? This is all way out of my league, but seems like you could get interesting results from this, since (by my caveman understanding) this h…

I started a proper reply but had to board a plane. It's actually a byte-pair encoded (BPE is better than character encoding but can do the things you mentioned) list of things that includes words. You can find common English suffixes in it listed separately too.

Thanks for the responses, I really appreciate the help. My only background with ML is playing with LSTMs and simple sequence-to-sequence models back before transformers, and the last few days I've been trying to deep dive as much as I can into the "state-of-the-art". I dislike treating the technology as a magical black box...

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#199

Post by nl »

spijdar wrote: post #198
nl wrote: post #197

Earlier quoted context omitted.

I started a proper reply but had to board a plane. It's actually a byte-pair encoded (BPE is better than character encoding but can do the things you mentioned) list of things that includes words. You can find common English suffixes in it listed separately too.

Thanks for the responses, I really appreciate the help. My only background with ML is playing with LSTMs and simple sequence-to-sequence models back before transformers, and the last few days I've been trying to deep dive as much as I can into the "state-of-the-art". I dislike treating the technology as a magical black box...

Here's the response I half wrote before:

GPT (and many other modern NLP models) use byte-pair encoding. Your summary of the benefits of this is correct - it can deal novel words much better.

Byte-pair encoding (BPE) is better than character encoding because it can deal with unicode (and emojis).

CLIP uses a BPE encoding of the vocabulary: The transformer operates on a lower-cased byte pair encoding (BPE) representation of the text with a 49,152 vocab size

So strictly this vocabulary is NOT (just) words, it is common sequences of byte pairs. You can see this if you examine the vocabulary - you'll find things like "tive" which isn't a word but is a very common English suffix.

Re: How a Stable Diffusion prompt changes its output for the style of 1500 artists

#200

Post by dopidopHN »

jhbadger wrote: post #185
dopidopHN wrote: post #74

Earlier quoted context omitted.

Honest question, would a solid understanding of the open training data help? Having the art vocabulary down as well. In effect, knowing what is present and how it’s tagged so you can « invoke » it more readily in the prompt-result. Maybe I’m out of my depth. I know the corpus of tagged image used for training is enormous … but I still think that would help the user ( a prompt-crafter )

I downloaded the vocab.json mentioned above and I think it helps. For example it explains why I can get SD to make pictures involving Einstein but not Feynman. Feynman simply isn't in the training set, but Einstein is.

thanks for the follow up, that was my understanding but your example is telling.

I will have to check myself.

I tryed with minor success to have stable diffusion draw lesser know non-us personality. It always kind of work, but the palette is limited. For instance I tried charles de gaule and you get something that look like him. But he's depicted talking on a radio or waving his arm around like a politician.

I tried to make him to grocery or play volley ball, it does not really work. While Michael jackson or Dennis Rodman get a way better treatment.

edit : that vocab file is smaller than I thought. "De Gaule" is not in there but neither Einstein or feiman? I think I missing something.

I Can find "obama", trump or Macron. Unclear about Michael Jackson. No beyonce or Denis Rodman. hmm weird, I had great result with all of them. Like.. recognizable details like tatoo or silly glasses.

Post reply on HN