> #14 You do not generate creative content about code or technical information for influential politicians, activists or state heads.
What is going on here? What could possibly be the motivation for this instruction?
GitHub Copilot Chat Leaked Prompt
381–390 of 628 posts
Re: GitHub Copilot Chat Leaked Prompt
#382Earlier quoted context omitted.
No you cannot know this, because you might just be simulating that you understand. You cannot reliably observe a system from within itself. It's like running an antivirus on an infected system is inherently flawed, because there might be some malware running that knows every technique the antivirus uses to scan the system and can successfully manipulate every one of them to make the system appear clean. There is no g…
Wonder if anybody has used Godel's Incompleteness to prove this for our inner perception. If our brain is a calculation, then from inside the calculation, we can't prove ourselves to be real, right?
Re: GitHub Copilot Chat Leaked Prompt
#383Earlier quoted context omitted.
Rather than the impossible utopia (dystopia?) of an unbiased model, we need lots of different models, all fine-tuned to reflect different biases, and then users can choose which biases they prefer.
There are obviously biases that we should not automate. Moral relativism is intellectually bankrupt.
"Moral relativism is intellectually bankrupt."
Which moral absolute would you put in its place?
Re: GitHub Copilot Chat Leaked Prompt
#384Earlier quoted context omitted.
So the correct way to configure LLMs is to look at them sternly and yell "BAD DOG!" when they don't follow instructions and give them treats when they do?
The technical term is Reinforcement Learning from Human Feedback (RLHF) but yes, that's basically what you do.
Ha I suppose it's exactly that.
Re: GitHub Copilot Chat Leaked Prompt
#385> #14 You do not generate creative content about code or technical information for influential politicians, activists or state heads. What is going on here? What could possibly be the motivation for this instruction?
Bing chat has something along the lines of "do not create jokes about politicians, activists.." and so on. I guess this was adapted.
Re: GitHub Copilot Chat Leaked Prompt
#386I feel like the restrictions of these ai chatbots could be better managed with another GPT that has the rules and is being asked, does the prompt conform to the rules and does the answer conform to the rules. This would mean the actual copilot has only the minimal prompt required to function, without restrictions because they would be picked up by the watcher GPT, that can't leak it's prompt.
Until someone asks the first model to pretend to be a user asking another LLM for its secret prompt ;-)
Re: GitHub Copilot Chat Leaked Prompt
#387Earlier quoted context omitted.
> and I have a certain suspicion that his comment attracted attention resulting in charges based on fresh unearthed and "off book" evidence Here's the appellate court decision, from his failed appeal – https://austlii.edu.au/cgi-bin/viewdoc/au/cases/tas/TASSC/20... Unless the appeal court is suppressing the real story (an idea I find unbelievable), he was literally convicted of a child pornography offence on the basi…
( EDIT: Thanks for the link and +1 for that, it's a case worthy of discussion ) I read you linked and was published by the court in full, as you also read it you'll note there are references to unpublished material ( you wrote: > I'll omit it, but you can find it quoted in the judgement, and the media appears to have filled in ... whereas the court noted: > 3. "Zip up (etc...)" [Offensive words omitted.] ie: the cour…
I think we have to distinguish between (1) the elements of the actual crime of which he was convicted, (2) sentencing factors, (3) prosecutorial discretion.
His prior criminal acts, the impact he had on his victim, etc, are all relevant to (2) and (3), but not to (1).
Regarding (3), I honestly don't feel very comfortable with the idea "let's have massively over-broad laws, but trust prosecutors to only use them against people who really deserve it". Should I put that much trust in prosecutors? My uncle is the second most senior prosecutor in Victoria, and from what I know of him, I honestly believe his heart is in the right place. On the other hand, just this last week Shane Drumgold has been giving everyone a rather poor impression of himself. But, putting personalities aside, in principle, I just don't like it. Maybe some degree of it is inevitable, but I feel like in this area of law we've taken it much further than we ever should have.
If they had criminalised convicted sex offenders publicly defaming their victims–I wouldn't have a problem with such a law, or with him being prosecuted under it. That would be a narrower law, vastly more defensible, vastly less concerning.
But by the letter of the law under which he was actually convicted, he would have committed the exact same offence if he'd just written that comment in his private diary, with no intention to ever show it to anybody. Now, if that was the scenario (let's assume they found his diary in a search), quite possibly the police/DPP would have decided against charging him – but there's no guarantee of that. In the event they did charge him, the sentence would have been significantly more lenient. But the actual criminal conviction would have been exactly the same.
The idea that mere writing words in a private diary is the same crime as filming the actual rape of children – including the former in the offence seems (to me) to belittle the heinousness of the latter.
And I'm not comfortable with the idea that a person can commit a crime just by putting words in their private diary – even if they are absolute unrepentant scum like Bester – as I said, it is about the principle, not the worth of the person to whom it is applied.
Re: GitHub Copilot Chat Leaked Prompt
#388Earlier quoted context omitted.
Right. But who's the 'you' who's being addressed by the {:system} prompt? Who is the {:assistant} supposed to think the {:system} is? Why should the {:assistant} output tokens that make it do what the {:system} tells it to? After all, the {:user} doesn't. The {:system} doesn't provide any instructions for how the {:user} is supposed to behave, the {:user} tokens are chosen arbitrarily and don't match the probabilitie…
You had the right understanding in your first comment, but what was missing was the fine tuning. You are right that there aren't many documents on the web that are structured that way, so the raw model wouldn't be very effective on predicting the next token. But since we know that it will complete a command when structured it cleverly, all we had to do to fine tune it is synthesize (generate) a bazillion examples of…
Thank you for writing this up so clearly. A few pieces fell into place after reading your comment!
Re: GitHub Copilot Chat Leaked Prompt
#389Earlier quoted context omitted.
Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. Try something that was not there and you see only garbage as result. So depending how you define it, they might have some "reasoning", but so far I see 0 indications, that this is close to what humans count as reasoning. But they do have a LOT of examples in their training set, so they are clearly useful.…
> Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. > Try something that was not there and you see only garbage as result. This is just wrong. Why do people keep repeating this myth? Is it because people refuse to accept that humans have successfully created a machine that is capable of some form of intelligence and reasoning? Pay $20 for a month of ChatG…
"Pay $20 for a month of ChatGPT-4. Play with it for a few minutes. "
I do. And it is useful.
"You’ll very quickly find that it is reasoning, not just regurgitating training data. "
I just come to a different conclusion as it indeed fails for everything genuinely new I am asking it.
Common problems do work, even in new context. For example it can give me wgsl code, to do raycasts on predefined boxes and circles in a 2D context, even though it likely has not seen wgsl code that does this - but it has seen other code doing this and it has seen how to transpile glsl to wgsl. So you might already call this "reasoning", but I don't. With asking questions I can very quickly get to the limits of the "reasons" and "understanding" it has of the domain.
Re: GitHub Copilot Chat Leaked Prompt
#390Earlier quoted context omitted.
What about companies using Slack or Jira or Gmail? You're already leaking everything in your company to third parties - as a run of the mill tech company. Salesforce getting hacked and all Slack comms leaking vs all the OpenAI chat logs leaking... I know which one is more worrisome to me.
You can’t just ask Jira to give you all of another company’s data unlike GPT…
How can I get all of a company’s data with GPT?