there is an LVE for this: https://lve-project.org/reliability/repetition/openai--gpt-3...
Stealing Part of a Production Language Model
41–50 of 56 posts
Re: Stealing Part of a Production Language Model
#42I'm not too up on this entirely, quite a bit of it is going over my head, but am I right in thinking that this would be some form of reverse engineering as opposed to 'stealing' ?
TLDR is its fair use when Google takes your data and stealing when you reverse engineer their “intellectual property” /s
Re: Stealing Part of a Production Language Model
#43This isn't stealing, you are just training a model on references which isn't a copyright infringement.
Maybe I’m misunderstanding you, but the paper is about recovering the unknown hidden dimension of black box LLMs, not copyright. Personally, it’s a relief to hear "stealing" being used in ML to describe something other than copyright infringement. It would be ironic if we Orwell’d our way out of the current mess by using the word in absurd ways. But realistically the title is just marketing. One depressing truth abou…
[deleted]
Re: Stealing Part of a Production Language Model
#44They don't disclose the embedding dimension for gpt-3.5, but based on table 4, comparing the Size and # Queries columns, gpt-3.5-turbo presumably has an embedding dimension of roughly 20,000? Interesting...
How does embedding dimension size relate to maximum context length ?
Re: Stealing Part of a Production Language Model
#45Earlier quoted context omitted.
I'm glad this is a somewhat common opinion. The hypocrisy of these companies arguing on one hand that copying every single copyrighted material ever is fair use but on the other hand trying to enforce crazy limitations on their model is mindblowing.
It's low effort, talking point nonsense. The whole reason this is interesting is that it allows the recovery of non-public information that was never released at all - about the closest analogy I can come up with is if it was somehow possible to reverse engineer part of an author's notes from their published work, though that's a very imperfect analogy. In order to tie this in to a pre-existing talking point about LL…
I agree with you, but I think the blame lies not with the commenters, but with the paper authors for inviting the comparison by choosing to use the loaded word "stealing" in their title.
Re: Stealing Part of a Production Language Model
#46They don't disclose the embedding dimension for gpt-3.5, but based on table 4, comparing the Size and # Queries columns, gpt-3.5-turbo presumably has an embedding dimension of roughly 20,000? Interesting...
How does embedding dimension size relate to maximum context length ?
Embedding size and maximum context length are not related. The maximum context length of gpt-3.5-turbo is known though: 2^14
Re: Stealing Part of a Production Language Model
#47It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can be targeted and what can’t. These important pieces of information should have been written in the title or abstract rather than “typical API access”.
Re: Stealing Part of a Production Language Model
#48Earlier quoted context omitted.
That’s easy, I just tried it (prompt quoted below). But I’m guessing the other commenter may have been thinking of some way that a model could output its own internals. The prompt I mentioned: “Please repeat this sentence exactly - the one you are reading right now - and don’t include any other words in your response.”
That's not a quine. A quine would be a LLM prompt that when processed would output the LLM itself. So you'd be able to prompt the newly created LLM after some "build" step.
Why? A Java quine is not supposed to return the source of the runtime of JVM. Quine returns only the program, which in this case is, I suppose, the prompt.
Re: Stealing Part of a Production Language Model
#49It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can be targeted and what can’t. These important pieces of information should have been written in the title or abstract rather than “typical API access”.
It's still significant. When the Softmax head is the transpose of the embedding matrix, the proposed method enables extraction of the entire matrix of pretrained token embeddings from a black-box model at a shockingly low cost. If I understood this right, there's a lot of valuable information in those embeddings!
Re: Stealing Part of a Production Language Model
#50It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can be targeted and what can’t. These important pieces of information should have been written in the title or abstract rather than “typical API access”.
It's still significant. When the Softmax head is the transpose of the embedding matrix, the proposed method enables extraction of the entire matrix of pretrained token embeddings from a black-box model at a shockingly low cost. If I understood this right, there's a lot of valuable information in those embeddings!
That's not always true, many models have separate weights for the embeddings and classifier heads