Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting AI models from mobile apps

altayakkus.substack.com

Post reply on HN
Print view View original

111–120 of 250 posts

Re: Extracting AI models from mobile apps

#111

Post by concerndc1tizen »

BobbyTables2 wrote: post #108
Workaccount2 wrote: post #106

Earlier quoted context omitted.

LLMs are not massive archives of data. They are a tiny fraction of a fraction of a percent of the size of their training set. And before you knee-jerk "it's a compression algo!", I invite you to archive all your data with an LLMs "compression algo".

Copying a single sentence verbatim from a 1000 page book is still plagiarism. And is technically copyright infringement outside fair use exceptions.

And similarly, translating those sentences into data points is still a derivative work, like transcribing music and then making a new recording is still derivative.

Re: Extracting AI models from mobile apps

#112

Post by philwelch »

salawat wrote: post #105
crazygringo wrote: post #98

Earlier quoted context omitted.

Well obviously not in general. But when it comes to copyright law specifically, yes absolutely. That is the question I'm asking.

You're not going to get an answer you find agreeable, because you're hoping for an answer that allows you to continue to treat the tool as chattel, without conferring to it the excess baggage of being an individuated entity/laborer. You're either going to get: it's a technological, infinitely scalable process, and the training data should be considered what it is, which is intellectual property that should be being l…

No, you’re missing the point of copyright. The point of copyright is to protect an exclusive right to copy, not the right to produce original works influenced by previous works. If an LLM produces original works that are influenced by the training data, that is not a violation of copyright. If it reproduces the training data verbatim, it is.

Re: Extracting AI models from mobile apps

#113

Post by drdeca »

Mordisquitos wrote: post #62
dragonwriter wrote: post #43

Earlier quoted context omitted.

> If companies train on data they don't own and expect to own their model weights, that's hypocritical. Its not hypocritical to follow a line of legal analysis whoch holds that copying material in the course of training AI on it is outside the scope of copyright protection (as, e.g., fair use in the US), but that the model weights resulting from the training are protected by copyright. It maybe wrong, and it may be c…

If the resulting AI models are protected by copyright that invalidates the claim that AI models being trained on copyrighted materials is fair-use analogous to human beings becoming educated by exposure to copyrighted materials. Educated human beings are not protected by copyright, hence neither should trained AI models. Conversely, if a copyrightable work is produced based on work which itself is copyrighted, the re…

If I take 1000 books and count the distributions of the lengths of the words, and the covariance between the lengths of one word and the next word for each book, and how much this covariance matrix tends to vary across the different books, and other things like this, and publish these summaries, it seems fairly clear to me that this should count as fair use.

(Such a model/statistical-summary, along with a dictionary, could be used to generate nonsensical texts which have similar patterns in terms of just word lengths.)

Should the resulting work be protected by copyright? I’m not entirely sure…

I guess one thing is, the specific numbers I obtain by doing this are not a consequence of any creative decision making on my part, which I think in some jurisdictions (I don’t remember which) plays a role in whether a work is copyrightable (I will use “copyrightable” as an abbreviation for “protected by copyright”. I don’t mean to imply a requirement that someone specifically registers for copyright.). (Iirc this makes it so phone books are copyrightable in some jurisdictions but not others?)

The particular choice of statistical analysis does seem like it may involve creative decision making, but that would just be about like, what analysis I describe, and how the numbers I publish are to be interpreted, not what the numbers are? (Analogous to the source code of an ML model, not the parameters.)

Here is another question: suppose there is a method of producing a data artifact which would be genuinely (and economically) useful, and which does not rely on taking in any copyrighted input, but requires a large (expensive) amount of compute to produce, and which also uses a lot of randomness so that the result would be different each time it was done (but suppose also that there isn’t much point doing it multiple times at the same scale, as having two of this kind of data artifact wouldn’t be much more valuable than having one).

Should such data artifacts be protected by copyright or something like it?

Well, if copyright requires creative human decision making, then they wouldn’t be.

It seems like it would make sense to want it to be economically incentivized to create such data artifacts of higher sizes (to a point of course. Only as much as is justified by the value that is produced by them being available.) .

If such data artifacts can always be distributed without restriction, then ones that are publicly available would be public goods, and I guess only ones that are trade secrets would be private goods? It seems to me like having some mechanism to incentivize their creation and being-eventually-freely-distributed would be beneficial?

But maybe copyright isn’t the best way to do that? Idk.

Re: Extracting AI models from mobile apps

#114

Post by baxtr »

int_19h wrote: post #107
Workaccount2 wrote: post #106

Earlier quoted context omitted.

LLMs are not massive archives of data. They are a tiny fraction of a fraction of a percent of the size of their training set. And before you knee-jerk "it's a compression algo!", I invite you to archive all your data with an LLMs "compression algo".

It doesn't matter. It's still a derived work.

Well what isn’t in this world?

Would Einstein would have been possible without Newton?

Re: Extracting AI models from mobile apps

#115

Post by dijksterhuis »

philwelch wrote: post #104
dijksterhuis wrote: post #94

Earlier quoted context omitted.

as a human being, and one that does music stuff, i don’t download terabytes of other peoples works from the internet directly into my brain. i don’t have verbatim reproductions of people’s work sitting around on a hard disk in my stomach/lungs/head/feet. LLMs are not humans. They’re essentially a probabilistic compression algorithm (encode data into model weights/decode with prompt to retrieve data).

Do you ever listen to music? Is your music ever influenced by the music that you listen to? How do you imagine that works, in an information-theoretical sense, that fundamentally differs from an LLM? Depending on how much music you've listened to, you very well may have "downloaded terabytes" of it into your brain. Your argument is specious.

i do listen to music.

i listen to it on apple music.

i pay money to apple for this.

some of that money that i pay to apple goes to the rights holders of that music for the copying and performance of their work through my speakers.

that’s a pretty big difference to how most LLMs are trained right there! i actually pay original creators some money.

-

i am a human being. you cannot reduce me down to some easy information theory.

an LLM is a tool. an algorithm. with the same random seed etc etc it will get the same results. it is not human.

you put me in the same room as yesterday i’ll behave completely differently.

-

i have listened to way more than terabytes of music in my life. doesn’t mean i have the ability to regurgitate any of it verbatim though. i’m crap at that stuff.

LLMs seem to be really good at it though.

Re: Extracting AI models from mobile apps

#116

Post by timewizard »

Workaccount2 wrote: post #106
Polizeiposaune wrote: post #39

You wouldn't train a LLM on a corpus containing copyrighted works without ensuring you had the necessary rights to the works, would you?

LLMs are not massive archives of data. They are a tiny fraction of a fraction of a percent of the size of their training set. And before you knee-jerk "it's a compression algo!", I invite you to archive all your data with an LLMs "compression algo".

> LLMs are not massive archives of data.

Neither am I, yet, I am still capable of reproducing copyrighted works to a level that most would describe as illegal.

> And before you knee-jerk "it's a compression algo!"

It's literally a fundamental part of the technology so I can't see how you call it a "knee jerk." It's lossy compression, the same way a JPEG might be, and simply recompressing your picture to a lower resolution does not at all obviate your copyright.

> I invite you to archive all your data with an LLMs "compression algo".

As long as we agree it is _my data_ and not yours.

Re: Extracting AI models from mobile apps

#117

Post by jpollock »

concerndc1tizen wrote: post #111
BobbyTables2 wrote: post #108

Earlier quoted context omitted.

Copying a single sentence verbatim from a 1000 page book is still plagiarism. And is technically copyright infringement outside fair use exceptions.

And similarly, translating those sentences into data points is still a derivative work, like transcribing music and then making a new recording is still derivative.

derivative works still tend to be copyright violations.

Re: Extracting AI models from mobile apps

#118

Post by _DeadFred_ »

philwelch wrote: post #89
Polizeiposaune wrote: post #39

You wouldn't train a LLM on a corpus containing copyrighted works without ensuring you had the necessary rights to the works, would you?

You’re applying a double standard to LLM’s and human creators. Any human writer or artist or filmmaker or musician will be influenced by other people’s works, even while those works are still under copyright.

Human creators don't store that 'influence' in a digital machine accessible format generated directly from the copyrighted content though.

Although with the 'good new everyone, we built the torment nexus' trajectory of AI my guess is at this point AI companies would just incorporate actual human brains instead of digital storage if that was the requirement.

Re: Extracting AI models from mobile apps

#119

Post by bayindirh »

FloorEgg wrote: post #102
bayindirh wrote: post #96

Earlier quoted context omitted.

The moment you earn money from it, that's not fair use anymore. When I last checked, unlimited access to said models were not free, plus it's not "research" anymore. - Addenda - For the interested parties, the law states the following [0]. Notwithstanding the provisions of sections 17 U.S.C. § 106 and 17 U.S.C. § 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or…

I really don't think it's that simple. I can read books and then earn money from applying what I learned in them. I can also study art and then make original art in the same or similar styles. If a person was doing this there would be no one claiming copyright infringement. The only difference is it's a machine doing it and not a person. The nature of copyright and plagiarism boils down to paraphrasing, and so long a…

> I can read books and then earn money from applying what I learned in them.

How many books can you read, understand and memorize in T time, and how many books an AI can ingest in the T time?

If we're down to paraphrasing, watch this video [1], and think again.

Many models, given that you ask the correct questions, reproduce their training set with great accuracy, and this is only prevented with monkey patching, IIUC.

So, it's still a big mess, even if we don't add copyrighted corpus to the mix. Oh, BTW, datasets like "The Stack" are not clean as they claim. I have seen at least two non-permissively licensed code repositories inside that dataset.

[1]: https://youtu.be/LrkAORPiaEA

Re: Extracting AI models from mobile apps

#120

Post by timewizard »

TechDebtDevin wrote: post #101
jonpo wrote: post #16

Well done you seem to have liberated an open model trained on open data for blind and visually impaired people. Paper: https://arxiv.org/pdf/2204.03738 Code: https://github.com/microsoft/banknote-net Training data: https://raw.githubusercontent.com/microsoft/banknote-net/ref... model: https://github.com/microsoft/banknote-net/blob/main/models/b... Kinda easier to download it straight from github. Its licenced under M…

[flagged]

[flagged]
Post reply on HN