Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things are about to get worse for generative AI

garymarcus.substack.com

Post reply on HN
Print view View original

621–630 of 769 posts

Re: Things are about to get worse for generative AI

#621

Post by aimor »

daemonologist wrote: post #545
tayo42 wrote: post #495

Earlier quoted context omitted.

This is a little confusing. You turned the text into indices? So numbers? Then compressed that? Or the text as numbers without any extra compression is only 1kb? The tokenizer the models use,(sentence piece) is more or less based on one way to do compression.(bpe). It's not really clear what your testing.

My reading is that at each generation step they ordered all possible next words by the probability assigned to them by the model and recorded the index of the true next word (so if the model was very good at predicting Harry Potter their indices would mostly be 0, 0, 0, ...).

This is correct

Re: Things are about to get worse for generative AI

#622

Post by noitpmeder »

RecycledEle wrote: post #599
iainctduncan wrote: post #493

Earlier quoted context omitted.

Copyright has never been based on a moral stance. It has always been determined by the lobbying power of various groups. The idea that we should dispense with it to let generative AI companies make even more money seems totally bizarre.

> The idea that we should dispense with it [copyright] to let generative AI companies make even more money seems totally bizarre. The idea is that we should remove abuses of copyright to allow our society to move forward, and thereby continue to exist. Imagine if there was a law at the beginning of the Industrial Revolution that said when non-human labor was used, the Animal Welfare Office had veto power. Then imagin…

This is an insane amount of fear mongering. Chinese shops have been shamelessly ripping IP from Western companies for years, should we now throw out those laws and let it happen in the US too for the sake of competitive advantage?

Why stop there? There's a ton of child labour in China and other part of the world that yield economic advantages. Should we let that happen in the western world too?

AI is wonderful in so many ways. But we should not throw out our entire way of life to adapt to a new technology.

Re: Things are about to get worse for generative AI

#623

Post by shkkmo »

ben_w wrote: post #614
shkkmo wrote: post #606

Earlier quoted context omitted.

This is full of so many inaccuracies. > Copyright is a fairly ancient concept The idea is fairly old, but it's current implementation in law is not nearly that old. > that benefits normal people as much as it benefits big corporations Clearly false if you measure that benefit in monetary terms. > Copyright is usually interpreted fairly narrowly, and doesn't prevent you from creating inspired works, or retelling the s…

> How? The restrictions on selling images of Mickey Mouse exist regarless of if they were created with or without AI assistance. Scale. GenAI automates creation of things that are derived from but strictly aren't the same as the original content; as it's (currently) not possible to automate the detection of derivative works (which is something copyright is supposed to be about), this means actual humans have to look…

> actual humans have to look at each case, and that's expensive and time consuming and O(n*m) on n new works that have to be compared against m existing in-copyright works for infringement.

That scale already exists. The amount of community generated derivative works already dwarfs the capacity of copyright holders to review each piece. The ease of publishing reproductions already makes endorcement a question of priorizing the larger infringers and ignoring those with no reach.

Indeed, prohibitions of training on copyrighted work without a special license seem like they make it harder to develop the sorts of AI can detect derivitave works.

As case law makes clear that people running the prompts and picking the output to keep are liable for infrinent then there will be demand for tools to detect derivitave works and either filter or warn the user.

Re: Things are about to get worse for generative AI

#624

Post by Spivak »

kranke155 wrote: post #365
DonsDiscountGas wrote: post #243

Earlier quoted context omitted.

I'm pretty sure if you tried to sell a CG Mario movie Nintendo would sue you into oblivion, and "the neural network did it" would not be considered a good defense by anybody, including the judge and jury.

Sure, but making it possible for the neural network to make the movie (eventually in seconds) is somehow ok? So people can make their own private CG Mario films, as long as they don't try to sell them? Here's my argument - even if the NN only makes the films for private consumption, eventually they'll be so widespread and fast at making them that won't matter, since everyone will be able to watch Mario movies of thei…

> So people can make their own private CG Mario films, as long as they don't try to sell them?

Yes, why not? This is just computer assisted private fanfic.

Re: Things are about to get worse for generative AI

#625

Post by 8organicbits »

CuriouslyC wrote: post #611
8organicbits wrote: post #570

Earlier quoted context omitted.

If I Xerox'd a book and sold copies to people I'm clearly violating copyright. I'm not sure I follow.

Nobody has given Xerox an injunction against researching or building copiers because you can copy books and sell them.

Right. If a publisher found a specific Xerox machine was being used to copy and commercially distribute a book, in violation of copyright, they'd ask for an injunction on the person doing that. With OpenAI, the NY Time can see their copyrighted material on both the input (training) side and distributed output (generated) side of a specific LLM implementation. So they cry foul on OpenAIs actions, not LLM in general.

There appears to be an open question about if the LLM can freely ingest copyrighted material and output it verbatim without violating copyright. That seems like an obvious "no" to me, unless we decide that LLM has special treatment.

Re: Things are about to get worse for generative AI

#626

Post by jcgrillo »

orange-mentor wrote: post #612
jcgrillo wrote: post #488

Earlier quoted context omitted.

I agree, and I would prefer to see concrete examples of LLMs being used productively and profitably in the industry in a "disruptive" manner--putting people out of work, etc--before we conclude they're somehow the next big thing. Basically, before claiming LLMs (or generative techniques, more generally) mean that we're on the doorstep of "general" intelligence, show me door! The outline of that door might look like i…

> ...nobody's actually doing this? I think you're right. I am a programmer and I use GPT occasionally, and I even pay 20 bucks a month (for now), but even for my job it's not a not a world-shattering improvement. > ... the entertainment value of typing things into the box and seeing what comes out ... I would only add that in a consumer society like ours, entertainment is important. Changes to entertainment seem to h…

> I am a programmer and I use GPT occasionally, and I even pay 20 bucks a month (for now), but even for my job it's not a not a world-shattering improvement.

I am also a programmer, and when I think about the amount of time I actually spend typing out code, even on a great day where all the stars have aligned just right and I can really bang out some code that's like... idk, 30-50% of my time? Usually it's much less, and I'm doing things like reading documentation, reading code, talking to people, etc. So it's hard to imagine Copilot or whatever making me much more effective at my job, as it can really only help with a fraction of it.

I could see someone making the assumption that being able to delegate programming tasks to a robot assistant might make them more productive, but often I find that I don't really understand a problem fully until I'm in the weeds solving it--by which I mean I haven't specified it completely until I've finished the implementation and written the tests. So I don't know to what extent being able to specify and delegate would really help me be more productive.

> having a bunch of little LLM-powered daemons buzzing around the internet is just gonna be freaky.

Yeah, they're not super cheap though so they need to get actual work done otherwise there's no reason to run them. Unlike blockchains, they don't have a pyramid scheme holding them up.

Re: Things are about to get worse for generative AI

#627

Post by shkkmo »

RecycledEle wrote: post #599
iainctduncan wrote: post #493

Earlier quoted context omitted.

Copyright has never been based on a moral stance. It has always been determined by the lobbying power of various groups. The idea that we should dispense with it to let generative AI companies make even more money seems totally bizarre.

> The idea that we should dispense with it [copyright] to let generative AI companies make even more money seems totally bizarre. The idea is that we should remove abuses of copyright to allow our society to move forward, and thereby continue to exist. Imagine if there was a law at the beginning of the Industrial Revolution that said when non-human labor was used, the Animal Welfare Office had veto power. Then imagin…

> Do you remember when they Hollyweird degenerates sued a Christian company that wanted to put our G-rated versions of the movies aimed at children? The Christian company never suggested they not pay for the movies. No matter what the Christian company was willing to pay, they were not allowed to publish child-friendly versions of the movies.

If anyone is curious, this is what is being semi-accuratly referenced: https://www.crosswalk.com/culture/features/editing-companies...

Whole I disagree that the motivation is "degeneracy" and I doubt that there isn't a sum large enough to get studios on board, it is a pretty interesting example to bring up when discussing how much control we should give copyright holders.

Notably, it is legal to have a filter that changes playback not legal to provide a modified version of the original, even if you paid for that original.

Re: Things are about to get worse for generative AI

#628

Post by WhiteNoiz3 »

kenmacd wrote: post #374
WhiteNoiz3 wrote: post #281

As I understood it, the legal precedent for generative AI is the same one that allows google to scrape websites in order to index them for search for the common good. Google also can display cached versions of websites which is the original content of those sites. No one is going to say that google is copyright infringement just because it is showing content from other websites verbatim. So I think this is a weak arg…

> I think generative AI should be able to provide links to similar source material in the training data Except these aren't databases, so that's generally not possible, in the same way that it's not possible for your provide links to the source material it took to write your reply. How much learning led to the weights on your neurons that allowed you to generate that? Where did you learn about using italics and it's…

> Except these aren't databases, so that's generally not possible

Not directly and not in every case, but it IS possible to use embeddings to link to similar material. People are doing it pretty commonly using the RAG approach and Bard is already providing sources, etc. It may not be perfect, but the onus is on the AI companies to figure out how to do it right not just claim helplessness.

> Okay... but then, if I write a book should I be able to opt out of you being allowed to read it? What conditions should I be able to put on who can read my work?

Sites that don't want to appear in search results or have sensitive info they don't want to get into search engines can use the Robots.txt which is as old as the internet. There are many valid reasons to have mechanisms to prevent something from being included in training data, and I would also argue this is a core feature that is necessary to spur adoption by businesses as we've already seen. Otherwise, I am not sure I understand your reasoning.. people can publish websites and opt to have them excluded from search, the same should apply to AI.

Re: Things are about to get worse for generative AI

#629

Post by noitpmeder »

tim333 wrote: post #596

They are just going to have to inform the AI in some sense of the current copyright situation and ask it not to infringe. It's the same for human writers. If you are writing an article for Wikipedia say, you should read relevant source articles and then rewrite in a way that isn't a copy and paste beyond a few words.

Ok I'll bite. Let's assume you've informed the current models about copyright and asked them not to infringe...

What happens when they continue to do so.

Re: Things are about to get worse for generative AI

#630

Post by koliber »

TheRoque wrote: post #405
koliber wrote: post #286

The responsibility for ensuring that copyrights were not violated fall on the person publishing the work. Whether they drew something themselves, hired an apprentice artists with no legal training to draw something, took a photograph of something, or used AI to create an image should not matter. Why does anyone assume that ChatGPT or other tools would NOT produce previously-copyrighted content? I can see a naive assu…

So it makes generative AI essentially unusable, because you don't know if the output is plagiarism or not, so you'd just doubt it always and never use it.

No. It”s still very helpful. However you can not blindly take whatever it produces and publish it.

Sometimes it hallucinates.

Sometimes it draws weird looking hands.

Sometimes it generates copyrighted materials.

Check the work it produces.

Post reply on HN