Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting AI models from mobile apps

altayakkus.substack.com

Post reply on HN
Print view View original

191–200 of 250 posts

Re: Extracting AI models from mobile apps

#191

Post by dragonwriter »

OkayPhysicist wrote: post #128
dragonwriter wrote: post #43

Earlier quoted context omitted.

> If companies train on data they don't own and expect to own their model weights, that's hypocritical. Its not hypocritical to follow a line of legal analysis whoch holds that copying material in the course of training AI on it is outside the scope of copyright protection (as, e.g., fair use in the US), but that the model weights resulting from the training are protected by copyright. It maybe wrong, and it may be c…

The model weights are the result of an automated process, by definition, and thus not protected by copyright. In my unusually well-informed on copyright but not a lawyer opinion, without any new legislation on the subject, I suspect that the most likely scenario for intellectual property rights surrounding AI is that using other people's works for training probably falls under fair use, since it's extremely transform…

> The model weights are the result of an automated process, by definition, and thus not protected by copyright.

Object code is the result of an automated process and is covered by the copyright on the source code.

Compilations are covered by copyright separate from that of the individual works, and it is arguable that a training set would be covered by a compilation copyright, and the result of applying an automated training processs to it would remain covered by that copyright.

Re: Extracting AI models from mobile apps

#192

Post by cess11 »

dang wrote: post #185
cess11 wrote: post #167

Earlier quoted context omitted.

What do you mean, "swipe"? The other person agreed they'd misjudged the article and apologised several hours before you wrote this.

"Does 'AI' pay your bills" was a gratuitous personal attack.

Is it? How? In your mind, does it imply some particular humiliation or something?

Re: Extracting AI models from mobile apps

#193

Post by [deleted] »

refulgentis wrote: post #178
mentalgear wrote: post #158

Earlier quoted context omitted.

Would be interesting if someone could detail the approach to decode the pre-post processing steps before it enters the model, and how to find the correct input encoding.

Boils down to "use Frida to find the arguments to the TensorFlow call beyond the model file" Key here is, a binary model is just a bag-of-floats with primitively typed inputs and outputs. It's ~impossible to write up more than what's here because either: A) you understand reverse engineering and model basics, and thus the current content is clear you'd use Frida to figure out how the arguments are passed to TensorFlo…

[deleted]

Re: Extracting AI models from mobile apps

#194

Post by philwelch »

cmiles74 wrote: post #164
philwelch wrote: post #104

Earlier quoted context omitted.

Do you ever listen to music? Is your music ever influenced by the music that you listen to? How do you imagine that works, in an information-theoretical sense, that fundamentally differs from an LLM? Depending on how much music you've listened to, you very well may have "downloaded terabytes" of it into your brain. Your argument is specious.

Information on how large language models are trained is not hard to come by, there are numerous articles that cover this material. Even a brief skimming of this material will make it clear that the training of large language models is materially different in almost every way from how human beings "learn" and build knowledge. There are still many open questions around the process of how humans collect, store, retrieve…

> There is little mystery to how large language models function and it's clear that their output is parroting back portions of their training data

If this were true, then you would be able to identify the specific work being "parroted" and you'd have a case for copyright infringement regardless of whether it was produced by an LLM at all. This isn't how LLMs work though. For instance, if an LLM's training data includes the complete works of a given author and then you prompt the LLM to write a story in the style of that author, it will actually write an original story instead of reproducing one of the stories in its training corpus. It won't be particularly good but it will be an original work.

It also isn't obvious whether or not, or to what degree, LLM training works differently from human learning. You yourself acknowledged that there are "many open questions" about how human learning works, so how can you be so confident that it's fundamentally different? It doesn't matter anyway because you can still apply the exact same standards to LLM output to judge whether it infringes copyright that you would to something that was produced by a human being.

Re: Extracting AI models from mobile apps

#195

Post by throwaway2037 »

salawat wrote: post #105
crazygringo wrote: post #98

Earlier quoted context omitted.

Well obviously not in general. But when it comes to copyright law specifically, yes absolutely. That is the question I'm asking.

You're not going to get an answer you find agreeable, because you're hoping for an answer that allows you to continue to treat the tool as chattel, without conferring to it the excess baggage of being an individuated entity/laborer. You're either going to get: it's a technological, infinitely scalable process, and the training data should be considered what it is, which is intellectual property that should be being l…

Hi. I like this post. There are some careful thoughts here.

Can you help me to understand the term "chattel" as you used it? I never heard the term before I read your post, and I needed to Google for it: (in general use) a personal possession.

(in law) an item of property other than freehold land, including tangible goods ( chattels personal ) and leasehold interests ( chattels real ). >>

Re: Extracting AI models from mobile apps

#196

Post by throwaway2037 »

philwelch wrote: post #112
salawat wrote: post #105

Earlier quoted context omitted.

You're not going to get an answer you find agreeable, because you're hoping for an answer that allows you to continue to treat the tool as chattel, without conferring to it the excess baggage of being an individuated entity/laborer. You're either going to get: it's a technological, infinitely scalable process, and the training data should be considered what it is, which is intellectual property that should be being l…

No, you’re missing the point of copyright. The point of copyright is to protect an exclusive right to copy, not the right to produce original works influenced by previous works. If an LLM produces original works that are influenced by the training data, that is not a violation of copyright. If it reproduces the training data verbatim, it is.

    > The point of copyright is to protect an exclusive right to copy, not the right to produce original works influenced by previous works.
As I understand, the definition of "the right to produce original works influenced by previous works" has been a slowly moving target in my lifetime. Think about the effects of the album Paul's Boutique by Beastie Boys. They went wild with sampling and paid very little (zero?) to license those samples. Then, there were a bunch of court cases in the US that decided that future samplers needed to license the samples from the original authors. However, the ability to create legal, derivative works is usually carefully defined in copyright law. Can you comment on this matter vis-a-via LLMs?

    > If an LLM produces original works that are influenced by the training data, that is not a violation of copyright.
I'm pretty sure if an LLM creates Paul's Boutique 2.0 in 2025 using incredible number of samples, then someone cannot sell it (or use it in a YouTube video) without first licensing those samples. I doubt very much someone could just "hide behind" an LLM and claim, "Oh, it is original, but derivative, work, created by an LLM." I doubt courts would allow that.

Re: Extracting AI models from mobile apps

#197

Post by throwaway2037 »

o11c wrote: post #100
bayindirh wrote: post #96

Earlier quoted context omitted.

The moment you earn money from it, that's not fair use anymore. When I last checked, unlimited access to said models were not free, plus it's not "research" anymore. - Addenda - For the interested parties, the law states the following [0]. Notwithstanding the provisions of sections 17 U.S.C. § 106 and 17 U.S.C. § 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or…

"Making money" does not immediately invalidate fair use, but it does wave a big red flag in the courts' faces.

I would be more nuanced on this matter. As I understand, in the US, fair use allows media to write critiques of cultural artefacts (sorry, I cannot think of a better, broad term). For example, you can include small quotes from the film script when writing a critique of it without requiring permission from the owner of the copyright. And, until the World Wide Web arrived to the masses in the mid-1990s, most critiques were published by commercial media outlets, such as a daily newspaper. They were certainly published by commercial, for-profit entities. That said, I think the intent of the fair use is very important to the courts, much more than the entity that is doing the fair use (newspaper, blogger, etc.).

Another weird carve-out for copyright law in the US: parody. Honestly, I don't know if other jurisdictions allow parody in the same protected manner.

Re: Extracting AI models from mobile apps

#198

Post by biosboiii »

JTyQZSnP3cQGa8B wrote: post #3

> Keep in mind that AI models [...] are considered intellectual property Is it ironic or missing a /s? I can't really tell here.

Hey, author here.

I understand that its not very clear if the neural net and its weights & biases are considered as IP, I personally think that if some OpenAI employee just leaks GPT-4o it isn't magically public domain and everyone can just use it. I think lawmakers would start to sue AWS if they just re-host ChatGPT. Not that I endorse it, but especially in IP and in law in general "judge law" ("Richterrecht" in german) is prevalent, and laws are not a DSL with a few ifs and whiles.

But it is also a "cover my ass" notice as others said, I live in Germany and our law regarding "hacking" is quite ancient.

Re: Extracting AI models from mobile apps

#199

Post by biosboiii »

hn8726 wrote: post #175
amolgupta wrote: post #160

For app developers considering tflite, a safer way would be to host the models on firebase and delete them when their job is done. It comes with other features like versioning for model updates, A/B tests, lower apk size etc. https://firebase.google.com/docs/ml/manage-hosted-models

That wouldn't help against the technique explained in the article, would it? Since the model makes it way into the device, it can be intercepted in a similar fashion. I'm not quite sure I understand the firebase feature btw. From the docs, it's pretty much file storage with a dedicated API? I suppose you can use those models for inference in the cloud, but still, the storage API seems redundant.

I think the comment author means offering inference via Firebase, with the model never leaving the backend.

This works, just like ChatGPT works, but has the downside of 1. You have to pay the computing for every inference 2. Your users can't access it offline 3. Your users will have to use a lot of data from their mobile network operator. 4. Your inference will be slower

And since SeeingAI infers the model every second, your and your customers bill will be huge.

Re: Extracting AI models from mobile apps

#200

Post by biosboiii »

jonpo wrote: post #16

Well done you seem to have liberated an open model trained on open data for blind and visually impaired people. Paper: https://arxiv.org/pdf/2204.03738 Code: https://github.com/microsoft/banknote-net Training data: https://raw.githubusercontent.com/microsoft/banknote-net/ref... model: https://github.com/microsoft/banknote-net/blob/main/models/b... Kinda easier to download it straight from github. Its licenced under M…

Author here, it would be nice to claim that I did this on purpose but I really did not know it was open source.

I was rather interested in the process of instrumenting of TF to make this "attack" scalable to other apps.

Post reply on HN