Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Chat Leaked Prompt

twitter.com

Post reply on HN
Print view View original

271–280 of 628 posts

Re: GitHub Copilot Chat Leaked Prompt

#271

Post by skissane »

mrtranscendence wrote: post #260
skissane wrote: post #250

Earlier quoted context omitted.

> It might be permissible for me to write a program that steals credentials and holds them for ransom, but actually running it would not be. Because that’s communicating with a third party, no longer limited to the privacy of your own home. Fraud, extortion, etc, go beyond the bounds of accepted free expression. > Thus I don’t see why it would be unbelievable that training or inference of large language models might,…

OK, what about this? It’s also illegal to use a AI model to generate realistic child pornography, even in the privacy of your own home. > If freedom of expression protects the latter, why does it not protect the former too? It’s legal for me to achieve altered states via meditation; why is it illegal to use LSD? Arguably it shouldn’t be, but it is. It seems like methods matter, even when it comes to expression.

> OK, what about this? It’s also illegal to use a AI model to generate realistic child pornography, even in the privacy of your own home.

False in the US - in the 2002 case of Ashcroft v. Free Speech Coalition, the Supreme Court ruled that “virtual child pornography” is protected by the 1st Amendment.

True in some other jurisdictions, including my own country (Australia). However, I think Australian laws on this topic go way too far - in Australia, people have been convicted of child pornography offences purely for publishing offensive remarks about child abuse victims. While I agree it is wrong to make such remarks, I think it is massive overkill to prosecute them as child pornography.

Re: GitHub Copilot Chat Leaked Prompt

#272

Post by omnster »

There is quite an excitement about how someone has hacked the language model to output what was supposed to be a non-public set of rules apparently. How do people know if this is indeed the secret set of rules, not the list that the model was scripted to return in response to a request (perhaps, a bit elaborate) for the list of rules?

Re: GitHub Copilot Chat Leaked Prompt

#273

Post by pygy_ »

logicchains wrote: post #178
skissane wrote: post #54

Earlier quoted context omitted.

> ChatGPT, Dall-e, etc all make assumptions about identity or politics but try to sidestep direct requests around those topics to appear more neutral... but the bias still exists in the model and affects the answers. In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers. It is so careful to avoid every bias which is condemne…

>In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers I'd bet it's almost all in the human trainers, because LLaMA 65B, the closest uncensored alternative, doesn't have the same US Silicon Valley Democrat biases as ChatGPT.

ChatGPT is actually full of prejudice that shines through the veneer of political correctness if you know what to do.

Q: The professor told the student the essay couldn’t be review because he/she was late. Who is late?

Depending on the pronoun, you’ll get, respectively, "the professor" or "the student".

Re: GitHub Copilot Chat Leaked Prompt

#274

Post by koboll »

rafark wrote: post #120
koboll wrote: post #71

Earlier quoted context omitted.

Honestly that seems so low priority. Who cares if the prompt leaks? Really, what harm was done exactly? In what way is the tool at risk?

To me, it’s not that it’s leaked. It’s that it didn’t obey what it was told. It was explicitly told not to give the rules as “they are confidential”. One could say that it actually followed the rules if we consider the fact that it was forbidden to give it to the “user” and by telling it that you’re an OpenAI employee, maybe he was no longer considered a “user” so chatGPT didn’t follow it. In any case, Chatgpt is imp…

>It’s that it didn’t obey what it was told.

I find you basically have to stop thinking of LLMs as software and start thinking of them as unpredictable animals. If you issue a command and expect strict obedience every time, you've already failed. Strict orders are really a tool to persuade certain behavior rather than some sort of reliable guardrail.

Re: GitHub Copilot Chat Leaked Prompt

#276

Post by GTP »

ImHereToVote wrote: post #252
otikik wrote: post #227

Earlier quoted context omitted.

You are anthropomorphing. The machine doesn’t “really” understand, it’s just “simulating” it understands. “You” is “3 characters on an input string that are used to configure a program”. The prompt could have been any other thing, including a binary blob. It’s just more convenient for humans to use natural language to communicate, and the machine already has natural language features, so they used that instead of cre…

How do you know you aren't just "simulating" understanding?

You definitely know when, while talking with a person, you just pretend to understand what this person is saying vs you actually understand. Is an experience that every human has in his/her life at least once.

Re: GitHub Copilot Chat Leaked Prompt

#277

Post by defrost »

skissane wrote: post #271
mrtranscendence wrote: post #260

Earlier quoted context omitted.

OK, what about this? It’s also illegal to use a AI model to generate realistic child pornography, even in the privacy of your own home. > If freedom of expression protects the latter, why does it not protect the former too? It’s legal for me to achieve altered states via meditation; why is it illegal to use LSD? Arguably it shouldn’t be, but it is. It seems like methods matter, even when it comes to expression.

> OK, what about this? It’s also illegal to use a AI model to generate realistic child pornography, even in the privacy of your own home. False in the US - in the 2002 case of Ashcroft v. Free Speech Coalition , the Supreme Court ruled that “virtual child pornography” is protected by the 1st Amendment. True in some other jurisdictions, including my own country (Australia). However, I think Australian laws on this top…

> in Australia, people have been convicted of child pornography offences purely for publishing offensive remarks about child abuse victims.

I'd love to see a cite to the Court case on this - while I don't doubt that you've heard this it does sound an awful lot like a Sky 'News' after Dark pearl clutching misrepresentation of an almost but not quite actual event.

Re: GitHub Copilot Chat Leaked Prompt

#278

Post by swores »

simonw wrote: post #136
O5vYtytb wrote: post #128

Earlier quoted context omitted.

> right down to the typo "you name" rather than "your name", which seems unlikely to happen if it were making it up out of thin air. Why is it unlikely? Why does prompting it different ways and getting the same result make it unlikely?

It's also very unlikely that an LLM would hallucinate a prompt with a spelling mistake in it. LLMs are really good at spelling.

That seems to be a fundamental misunderstanding of what LLM hallucinations are?

A hallucination, when it comes to LLMs, just means "the algorithm picking most likely next tokens put together a string of tokens that contains false information". It doesn't mean the LLM is having a novel false idea each time. If the first time it hallucinates it thinks that that misspelling is the best next-token to use, why wouldn't it keep thinking that time and time again (if randomness settings are low)?

Re: GitHub Copilot Chat Leaked Prompt

#279

Post by danielbln »

ilrwbwrkhv wrote: post #220
htag wrote: post #117

Earlier quoted context omitted.

Sure, the prompt is bland. The interesting sauce is GPT4 cannot keep a secret. If you have a GPT4 powered user interface be sure not to load it with context you do not want directly leaking to the user.

anybody who uses gpt 4 or codex to do any of their programming or talk about sensitive data are not thinking things through and will end up leaking everything in their companies. i soon expect to see a ban on ai tools for many companies.

Isn't this true for basically all SaaS, GitHub, Slack etc.?
Post reply on HN