Live data from Hacker News

Stealing Part of a Production Language Model

arxiv.org

11–20 of 56 posts

Re: Stealing Part of a Production Language Model

#11

Earlier quoted context omitted.

Do you consider blind SQL injection an attack? If a user is not meant to have access to information, but they can somehow access it, that is a vulnerability.

Sure, except the information here is not other users' account info or something, it's analogous to the source of the DBMS. Which is not typically considered sensitive.

Tell that to Oracle…

Re: Stealing Part of a Production Language Model

#12

The implications of this sentiment are disturbing. It is considered an "attack" to probe at something to understand how it works in detail. In other words, how basically all natural science is done. What the fuck has this world turned into?

Attack here is a term specific to cryptography and infosec, it does not imply doing anything illegal or violent. "Oracle attack" is the more specific term where you probe a system to give up some of its internals it was not meant to expose.

Re: Stealing Part of a Production Language Model

#13

Earlier quoted context omitted.

Do you consider blind SQL injection an attack? If a user is not meant to have access to information, but they can somehow access it, that is a vulnerability.

Sure, except the information here is not other users' account info or something, it's analogous to the source of the DBMS. Which is not typically considered sensitive.

No, it's like the schema of the database which is normally a secret.

Re: Stealing Part of a Production Language Model

#18
post #14
post #2

Note: Google did not release the hidden dimension for GPT-3.5, and OpenAI has already implemented mitigations against some of this.

> Google did not release the hidden dimension for GPT-3.5 Google?

They may have meant that deepmind is a good majority of the authors, though ETH Zurich, a few others and even openai is represented as well. Does seem like strange wording if the paper went read though

Re: Stealing Part of a Production Language Model

#19
post #5

The implications of this sentiment are disturbing. It is considered an "attack" to probe at something to understand how it works in detail. In other words, how basically all natural science is done. What the fuck has this world turned into?

OpenAI has blatantly said that the "open" in their name was a deceptive marketing ploy. At this point in time, they aren't interested in sharing much if any real research, and trying to discover this information is now an "attack" against them.

OpenAI didn't write this paper.
Post reply on HN