Live data from Hacker News

Stable Diffusion 2.0

stability.ai

361–370 of 519 posts

Re: Stable Diffusion 2.0

#361
post #325

Earlier quoted context omitted.

> It would be interesting to see if this technology can erode the copyright concept a bit Copyright law (especially in US) only ever changes in the direction that suits corporations. So - no. What I expect instead is artists being sued by a big tech company for copyright violations because that big tech company used the artist Public Domain image for training their copyrighted AI and as a result it created a copyrigh…

In this situation, it would seem like the suit would end up at "comparing the timestamp at which the public domain and copyrighted versions were published", wouldn't it ? There is nothing that the generative AI can do in this process that's legally different from copy pasting the image, editing it a bit by hand, and somehow claiming intellectual property of the _initial_ image, no ?

In theory yes, in practice you have to pay your legal expanses in US even if you win the case. Which means you can bankrupt because a big company thought you infringed on their rights even if you didn't. Simply because you can't afford the costs.

It's absurd.

Re: Stable Diffusion 2.0

#362
post #81

Earlier quoted context omitted.

To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned"…

> when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". This is just lossy compression with a large and well-tuned (to the expected problem domain) dictionary. Video compression codecs can achieve a 500x compression ratio, and they are general-purpose.

human memory is lossy compression

Re: Stable Diffusion 2.0

#363
In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat.

1. https://twitter.com/emostaque/status/1595731407095140352?s=4...

Re: Stable Diffusion 2.0

#364

Earlier quoted context omitted.

Ah I am glad to see someone else talking about using public domain images! Honestly it baffles me that in all this discussion, I rarely see people discussing how to do this with appropriately licensed images. There are some pretty large datasets out there of public images, and doing so might even help encourage more people to contribute to open datasets. Also if the big ML companies HAD to use open images, they would…

Human artists derive their inspiration and styles from a large set of copyrighted works, but they are free to produce new art despite of that. Art would have developed much slower and be much poorer if, for example, Impressionism or Cubism had been entangled in long ownership confrontations in courts. Then there's the fact that humanity has been able to develop and share art and literary works for thousands of years…

That is a very apropos reference. If you're familiar with Cubism, you know that there's Picasso, and then there's Braque. The one is an art celebrity beyond almost any other, and the other isn't.

But they developed Cubism in parallel. There were periods where their work was almost indistinguishable. "Houses at l'Estaque", the trope namer for Cubism thanks to the remarks of a critic, was in fact by Braque.

You can generate infinite recognizable Basquiat from an AI, but is it Basquiat? No, of course not, because Basquiat's style operates within the context of a specific individual human making a point about expectations and the interface between his race and his artistic boldness and audacity as experienced by his wealthy audience. Making an AI 'ape' (!) his art style is itself quite the artistic statement, but it's not the same thing in the slightest.

You can generate infinite Rothko as 512x512 squares, but if you don't understand how the gallery hangings work and their ability to fill your entire visual field with first carefully chosen color, and then a great deal of detail at the threshold of perception of distinctions between color shades meant to further drive home the reaction to the basic color's moods, what you generate is basically arbitrary and nothing. Rothko isn't 'just a random color', Rothko is about giving you a feeling through means that aren't normal or representational, and the unusualness of this (reasonably successful) effort is what gave the work its valuation.

Ownership of the experience by a particular artist isn't the point. Rothko isn't solely celebrity worship and speculation. Picasso isn't all of Cubism. Art is things other than property of particular artists.

What makes it awkward is the great ease by which AI can blindly and unhelpfully wear the mask of an artist, such as Basquiat, to the detriment of art. It's HOW you use the tools, and it's possible to abuse such tools.

Re: Stable Diffusion 2.0

#365

To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned"…

  >> it can't memorize 240TB of data, so it "learned"
learning is a form of memorization but yeah

Re: Stable Diffusion 2.0

#366

Earlier quoted context omitted.

There are already laws against it but enforcement is laking, as always.

If you can't process/digest copyrighted content with algorithms/machine learning then Google Search (the whole thing, not just Image Search) is dead. So no, it's not at all clear where the legal lines are drawn. There have been no court cases yet, regarding the training of ML models. People are trying to draw analogies from other types of cases, but this has not been tried in court yet. And then the answer will likel…

[deleted]

Re: Stable Diffusion 2.0

#367

Is there a good explanation of how to train this from scratch with a custom dataset[0]? I've been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of stra…

> Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. Doesn't the "BY" part of the license mean you have to provide attribution along with your models' output[0]? I feel you'll have the equivalent of Github Copilot problem: it might be prohibitive to correctly attribute each output, and listing the entire dataset in attribution…

I feel quite strongly that there is a large difference between Stable Diffusion and Copilot: with the size of the training set vs the number of parameters, it should be very difficult if not impossible for Stable Diffusion to memorize and, by extension, copy paste to produce its outputs. Copilot is trained on text and outputs text. Coding is also inherently more difficult for an AI model to do. I expect it will memorize large portions of its input and is copy pasting in many cases to produce output. I therefore believe Copilot is doing "copyright laundering" but Stable Diffusion is not. Furthermore, I do not believe, for example, that artists should be able to copyright a "style" - but I would like to see them not be negatively impacted by this. Its complicated.

Re: Stable Diffusion 2.0

#368
post #363

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat. 1. https://twitter.com/emostaque/status/1595731407095140352?s=4...

this is really disappointing

Re: Stable Diffusion 2.0

#369
post #363

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat. 1. https://twitter.com/emostaque/status/1595731407095140352?s=4...

does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change

Re: Stable Diffusion 2.0

#370

Is there a good explanation of how to train this from scratch with a custom dataset[0]? I've been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of stra…

> Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. Doesn't the "BY" part of the license mean you have to provide attribution along with your models' output[0]? I feel you'll have the equivalent of Github Copilot problem: it might be prohibitive to correctly attribute each output, and listing the entire dataset in attribution…

The SA part (ShareAlike) is even more restrictive, as it imposes a license on the derivative work.

"— If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original"

Post reply on HN