Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Copilot Chat Leaked Prompt

twitter.com

Post reply on HN
Print view View original

291–300 of 628 posts

Re: GitHub Copilot Chat Leaked Prompt

#291

Post by pharmakom »

xpe wrote: post #110
flangola7 wrote: post #61

Earlier quoted context omitted.

There are obviously biases that we should not automate. Moral relativism is intellectually bankrupt.

> moral relativism is intellectually bankrupt I practically agree in the probabilistic sense. I don’t think I’m willing to categorically dismiss moral relativism, but I find it very unpersuasive at best, delusional in most cases, and dishonest at worst. Here’s one reason. Say you find a person who claims to be a moral relativist. It is always possible to present them with an ethical dilemma where they have to choose.…

You misunderstand moral relativism.

Moral relativists can have moral beliefs. They simply recognise that these are their own (very strongly held) opinions, rather than a reference to a universal moral code (like say, the Ten Commandments)

Re: GitHub Copilot Chat Leaked Prompt

#292

Post by scoot »

silverwind wrote: post #49
afro88 wrote: post #25

Earlier quoted context omitted.

But also, how would it even know if the code is copyrighted?

Microsoft has very precise tools like the licensee ruby gem to determine a repo's license which I'm sure their bot is aware of while training on said repo.

You can't determine a repository's licence, because a (mono)repo may contain multiple projects, each potentially under a different license,

Re: GitHub Copilot Chat Leaked Prompt

#294

Post by tanepiper »

jameshart wrote: post #109

Something that I find weird about these chat prompts (assuming they are real, not hallucinated): They're almost always written in second person*. "You are an AI programming assistant" "You are about to immerse yourself into the role of another Al model known as DAN" Who are these prompts addressed to? Who does the GPT think wrote them? The thing that confuses me is that these are text token prediction algorithms, und…

When I built https://botsin.space/@StochasticEntropy I wasn't actually sure if I had found an exploit where it was returning responses to other people's questions - but OpenAI assure me it's completely random stochastic hallucinations.

But most of the replies are the AI is responding in the first person to a question it was never asked, but it knows it's an AI agent and will sometimes tell us that.

(FWIW I usually start my code or refactoring requests with a "please" - it's not that I think it'll find it rude, but I think it's just how I was taught manners)

Re: GitHub Copilot Chat Leaked Prompt

#295

Post by jiggawatts »

pygy_ wrote: post #273
logicchains wrote: post #178

Earlier quoted context omitted.

>In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers I'd bet it's almost all in the human trainers, because LLaMA 65B, the closest uncensored alternative, doesn't have the same US Silicon Valley Democrat biases as ChatGPT.

ChatGPT is actually full of prejudice that shines through the veneer of political correctness if you know what to do. Q: The professor told the student the essay couldn’t be review because he/she was late. Who is late? Depending on the pronoun, you’ll get, respectively, "the professor" or "the student".

GPT 4 says that the student was late. If you look at the sentence objectively, then the best interpretation is that the submission was late, not that the "professor turned up late", since essays are typically reviewed over multiple days, not at a specific date and time where lateness of the reviewer might be a factor.

Sometimes GPT 4 says that the sentence is ambiguous, but as I explained above, it isn't really, but that takes a lot of understanding of the review process.

Re: GitHub Copilot Chat Leaked Prompt

#296

Post by radres »

htag wrote: post #117
gkoberger wrote: post #91

I feel like we've put too much emphasis on the prompts, as though they're some sort of special sauce. In reality, though, they're all pretty bland. It's like getting ahold of an employee handbook for Applebees. It feels scandalous to see the inner workings, because we're not supposed to see it, but ultimately it's basically what you would have guessed anyway.

Sure, the prompt is bland. The interesting sauce is GPT4 cannot keep a secret. If you have a GPT4 powered user interface be sure not to load it with context you do not want directly leaking to the user.

I made a twitter bot and specifically told the bot to act like a human, and be bit of an asshole (to gain more interactions). Then someone got angry and asked "Who are you!!!" and my bot replied "I am an AI language model..." I mean, come on GPT.

Re: GitHub Copilot Chat Leaked Prompt

#297

Post by wouldbecouldbe »

skissane wrote: post #151
emodendroket wrote: post #141

Earlier quoted context omitted.

I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.”

> I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.” OpenAI didn't just fine-tune it to avoid blatant racial/sexist/etc content, they openly claim to have invested a lot of effort in fine-tuning it to avoid subtle biases in those areas. And to be honest, a lot of people do feel…

If you read the comments here, the complaints are either to PC & US centric, or to sexist.

Reads to me like there is no way to win it for openai, there is going to be people complaining.

I use it a lot for code, and at times to summarize research on certain topics, but never run into any weird feedback, so depends on your usage.

Re: GitHub Copilot Chat Leaked Prompt

#298

Post by numpad0 »

JieJie wrote: post #169
skissane wrote: post #54

Earlier quoted context omitted.

> ChatGPT, Dall-e, etc all make assumptions about identity or politics but try to sidestep direct requests around those topics to appear more neutral... but the bias still exists in the model and affects the answers. In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers. It is so careful to avoid every bias which is condemne…

I wonder if it's as simple as adding a location check and updating the systemPrompt to tell the bot to assume the user is from _location_? How realistic is it if one says, "Hi, Bing. I'm from East London, please talk to me like you would to another East Londoner." Or is it obnoxiously stereotypical? I imagine, properly trained, that is something LLMs would excel at, communicating with people in their preferred mode o…

Will the conclusions reached by LLM always coincide with that of baseline American, or should they also understand and follow East Londonian logic?

Re: GitHub Copilot Chat Leaked Prompt

#299

Post by KeplerBoy »

gedy wrote: post #85
koboll wrote: post #71

Earlier quoted context omitted.

Honestly that seems so low priority. Who cares if the prompt leaks? Really, what harm was done exactly? In what way is the tool at risk?

> Who cares if the prompt leaks? I wonder if people are just overestimating how valuable "prompt engineering" is, and thinking it's some secret sauce or IP.

Isn't it rumored that midjourney's secret sauce is the way they optimize the user-provided prompt into a better prompt before feeding it into their version of Stable Diffusion?

They also don't have to worry about the model leaking it's prompt.

Re: GitHub Copilot Chat Leaked Prompt

#300

Post by qup »

mrtranscendence wrote: post #243
simonw wrote: post #123

Here's why I don't think this leaked prompt is hallucinated (quoting from my tweets https://twitter.com/simonw/status/1657227047285166080 ): Any time something like this happens a bunch of people suspect that it might be a hallucination, not the real prompt I used to think that but I don't any more: prompt leaks are so easy to pull off, and I've not yet seen a documented case of a hallucinated but realistic leak One…

> and I've not yet seen a documented case of a hallucinated but realistic leak How would you know? As far as I know no company has come out and confirmed that any purportedly leaked prompts are genuine.

Would it be more relevant to try to guess if the past prompts were perfectly accurate?

Or just give it your own prompt, extract that secret, and compare it directly to your own source?

Post reply on HN