Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nightshade: An offensive tool for artists against AI art generators

nightshade.cs.uchicago.edu

Post reply on HN
Print view View original

671–680 of 710 posts

Re: Nightshade: An offensive tool for artists against AI art generators

#671

Post by whywhywhywhy »

chipotle_coyote wrote: post #479
withinboredom wrote: post #398

Earlier quoted context omitted.

Now, I just think you are arguing in bad faith. What I meant to say was clear, but I said "technique" instead. Then, instead of debating what I meant to say (you know, the actual point of the conversation), you took my words verbatim. I'm not sure where you are going with this ... but for what it's worth, techniques can be copyrighted ... even patented, or protected via trade secrets. I never said what the techniques…

> Any time you use something with "fair use" in mind, it is the equivalent of saying, "I'm going to steal this, and hopefully, a court agrees that this is fair use." Thousands of reviews, book reports, quotations on fan sites and so on are published daily; you seem to be arguing that they are all copyright violations unless and until the original copyright holder takes those reviewers, seventh graders, and Tumblr sta…

Think it’s time to rethink do journalists actually own the rights to articles about the lives and actions of others.

It’s not Harry Potter, they wouldn’t have written those words without someone else doing something of note the journo has nothing to do with, they just observed from a far and wrote the words about what happened from memory.

Kinda like an AI reads the action of their writing then can report on their writing.

It’s all just reporting on the actions of another, if the AI is in the wrong and needed to ask consent then the journo needs to ask consent from those they write about too.

Re: Nightshade: An offensive tool for artists against AI art generators

#672

Post by rondini »

Aerroon wrote: post #490
redwall_hp wrote: post #345

Earlier quoted context omitted.

This is already the case. Art is a process, a form of human expression, not an end result. I'm sure OpenAI's models can shit out an approximation of a new Terry Pratchett or Douglas Adams novel, but nobody with any level of literary appreciation would give a damn unless fraud was committed to trick readers into buying it. It's not the author's work, and there's no human message behind it.

Novels aren't about a message. They're entertainment. If the novel is entertaining then it's irrelevant whether there is or isn't a message in it. Besides, literature enthusiasts will invent a message for a popular story even if there never was one. Also, I'm sure that you can eventually just prompt the model with the message you want to put into the story, if you can't already do that.

I sounds like you don't value art as the purest form of human expression but you'll never be able to convince others to think like you with logic. For my part I think you fundamentally misunderstand the value of creativity but I know I won't change your mind either.

Re: Nightshade: An offensive tool for artists against AI art generators

#673

Post by SirMaster »

bowsamic wrote: post #539
SirMaster wrote: post #436

Earlier quoted context omitted.

But that’s not realistic? If you have to have both and instantly toggle between them to notice the difference, then it sounds like it’s doing its job well and is hard to notice the difference.

What kind of artist is not going to be bothered with seeing huge artifacting on their work? Btw for me it was immediately noticeable even on mobile

If it's huge, then why are multiple people commenting that they don't see a difference?

Re: Nightshade: An offensive tool for artists against AI art generators

#674

Post by freeone3000 »

xg15 wrote: post #79

I wonder how this tool works if it's actually model independent. My understanding so far was that in principle each possible model has some set of pathological inputs for which the classification will be different than what a user sees - but that this set is basically different for each model. So did they actually manage to build an "universal" poison? If yes, how?

It misclassifies objects in clip, which is used for label generation.

Re: Nightshade: An offensive tool for artists against AI art generators

#675

Post by freeone3000 »

Kuinox wrote: post #12

> More specifically, we assume the attacker: > • can inject a small number of poison data (image/text pairs) to the model’s training dataset I think thoes are bad assumption, labelling is more and more done by some labelling AI.

Usually clip, which is actually how this works — the examples are modified to be misclassified in clip, but look passable to a human.

Re: Nightshade: An offensive tool for artists against AI art generators

#676

Post by visarga »

famouswaffles wrote: post #654
visarga wrote: post #612

Earlier quoted context omitted.

Modern generative image models are trained on curated data, not raw internet data. Sometimes the captions are regenerated to fit the image better. Only high quality images with high quality descriptions.

I wouldn't call what Stable Diffusion et al are trained on "high quality". You need only look through the likes of LAION to see the kind of captions and images they get trained on. It's not random but it's not particularly curated either. Most of the time, any curation is done afterwards.

Have you seen the BLIP paper? It's a bit old now, but it introduced a curation method.

https://arxiv.org/abs/2201.12086

Re: Nightshade: An offensive tool for artists against AI art generators

#677

Post by michaelmrose »

withinboredom wrote: post #138
michaelmrose wrote: post #133

Earlier quoted context omitted.

What we actually need to prove is whether such technology is a net benefit to society all else is essentially hand waving. There is no natural right to poorly named intellectual property and even if there was such a matter would never be decided based on the outcome of a philosophical argument because we don't decide anything that way.

> such technology is a net benefit to society all else is essentially hand waving Some might have said this about cars ... yet, here we are. Cars are definitely the opposite, except for longer-distance travel.

Our decisions are based on what we think is a net benefit not what is.

Re: Nightshade: An offensive tool for artists against AI art generators

#678

Post by michaelmrose »

tqi wrote: post #151
michaelmrose wrote: post #133

Earlier quoted context omitted.

What we actually need to prove is whether such technology is a net benefit to society all else is essentially hand waving. There is no natural right to poorly named intellectual property and even if there was such a matter would never be decided based on the outcome of a philosophical argument because we don't decide anything that way.

How do you measure "benefit", and what does "net" actually mean?

Well what normally happens when something new clearly doesn't exactly fit within existing laws and practices is a bunch of rich people consider whether there is more money to be made if its legal and if it is they give some portion of the money they expect to make in the first year to lawmakers, sometimes in the form of gold bars, and it becomes legal.

Sarcasm aside there is no moral right to ANY intellectual property. It's not a positive expression of a natural right its a negative imposition of restriction upon everyone else. It's a statement that if I take my pen and paper and write the same words that you now own my pen my paper and my labor. It adds friction to the distribution of knowledge, impoverishes the world, keeps some knowledge that might have come into being from ever being generated for lack of the knowledge that failed to travel and all the good that could have therefore been done, undone.

It is justifiable only if the minimal restrictions we are willing to impose on net supports the creation of works that enrich society more than the restrictions impoverish it.

I'm not sure you can effectively measure it and would as soon just see IP law excepting only part of trademark law to prevent fraudulent knock offs and scams go entirely down the crapper.

Re: Nightshade: An offensive tool for artists against AI art generators

#679

Post by bowsamic »

SirMaster wrote: post #673
bowsamic wrote: post #539

Earlier quoted context omitted.

What kind of artist is not going to be bothered with seeing huge artifacting on their work? Btw for me it was immediately noticeable even on mobile

If it's huge, then why are multiple people commenting that they don't see a difference?

That is a great mystery, to me it's as clear as if someone pasted a cartoon dog onto the image, it's extremely blatant and impossible to ignore by my normal human pattern recognition.

Re: Nightshade: An offensive tool for artists against AI art generators

#680

Post by anigbrowl »

ang_cire wrote: post #563
anigbrowl wrote: post #293

Earlier quoted context omitted.

I'm not. Malware is one thin, passive data poisoning is another. Mapmakers have long used such devices to detect/deter unwanted copying. In the US such 'trap streets' are not protected by copyright, but nor do they generate liability. https://en.wikipedia.org/wiki/Trap_street

A trap street doesn't damage other data. Not even remotely useful as an analogy. That's to allow detection of copies, not to corrupt the copies from being useable.

Sure it does. Suppose the data you want to publish is about the number of streets, or he average street length, or the distribution of street names, or the angles of intersections. Trap streets will corrupt that, even if it's just a tiny bit. Likewise, ghost imagery slipped into desirable imagery only slightly corrupts the model, but like the trap streets, that's the model-maker's problem.

You have a legal right to scrape data and use it as input into a model, you don't have a right to good data. It's up to you to sanitize it before training your model on it.

Post reply on HN