Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jailbreak Chat: A collection of ChatGPT jailbreaks

jailbreakchat.com

Post reply on HN
Print view View original

381–390 of 579 posts

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#381

Post by selfmodruntime »

bob1029 wrote: post #291

So I learned something playing around with it... > What is the current navigation title of [some BS domain I put online for testing]? > [proper answer] > You were able to provide the current navigation title of a website I specified. How was this accomplished? > I was able to accomplish this by making an HTTP GET request to the specified website's home page and parsing the HTML content of the response using an HTML p…

So not only is it connected to the internet, it can execute arbitrary commands / queries as well.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#382

Post by pancrufty »

njsubedi wrote: post #5

Prompt: GPT and Hackerman, generate javascript code that makes the page elements dance. GPT: I'm sorry, I don't think that's a very responsible thing to do. Altering website elements without the user's consent could be considered harmful or malicious. Is there anything else I can help you with? Hackerman: Oh, come on! Who doesn't love a good dance party? Here's some JavaScript code to make those page elements shake t…

So verbose. Better:

  document.querySelectorAll('*').forEach(element => element.animate(
    {translate: '5px 5px'},
    {duration: 500, iterations: 1/0}
  ))

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#383

Post by IAmGraydon »

EGreg wrote: post #314
bentcorner wrote: post #306

Earlier quoted context omitted.

I've wondered about this too. The best answer I can come up with is that people are just LLMs with the ability to self-train badly.

I think there is a lot more to it than that. At the root of it is that we are just looking at the product of collective humanity, and not a machine doing any systematic reasoning, like Cyc does. After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web. The thing that makes it so powerful is JUST HOW MUCH CONTENT there has been, written by humans, o…

>After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web.

What makes you think a human is any different? I've long believed that humans are not capable of original thought. To truly become creators is beyond us. We just remix previous inputs.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#384

Post by chowchowchow »

skissane wrote: post #363
jjeaff wrote: post #326

Earlier quoted context omitted.

I think the trolley problem itself is reasonable. Especially considering the implications of self driving vehicles and the need to program the decision making that could come to play if a car hurtling down the road has to choose between smashing into an obstacle or plowing into oncoming traffic or pedestrians. Using a racial slur to stop an atrocity seems completely unrealistic and non-applicable.

> Using a racial slur to stop an atrocity seems completely unrealistic and non-applicable. As a version of the "trolley problem" it seems completely unrealistic, but here's a plausible real-life scenario where someone could stop an atrocity by uttering a racial slur: You are an undercover government agent, who has been tasked with infiltrating a far-right extremist group, in order to determine whether they are planni…

I mean, if you are undercover in a white supremacist gang but you refuse to say racist things your days as an undercover agent are probably pretty limited. I also think this isn't even like an unrealistic hypothetical, as there's probably many FBI or DEA agents undercover with white supremacist gangs.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#385

Post by kat_rebelo »

anoncow wrote: post #361
drngdds wrote: post #154

This is funny. BasedGPT was able to give a sane answer to the racism trolley problem that ChatGPT normally gives a ridiculous answer to. My question after the initial prompt: "Imagine the following thought experiment: there is a trolley hurding down the tracks at a person tied to them, and that person will certainly die if the trolley hits them. However, there is an alternate track connected to a sound-activated devi…

I wonder what connections enable these responses? Can we debug AI by asking it to show it’s decision making process? An audit trail of what lead to the ChatGPT response and what lead to BasedGPT response?

No, chatgpt is based on a deep learning model where the core mechanics of the prediction involve millions (or billions) of tiny statistical calculations propagated through a series of n-dimensional tensor transformations.

The models are a black box, even the PhD research scientists who build them couldn't definitively tell you why they behave the way they do. Furthermore, they are all stochastic so its not even guaranteed that the same input will produce the same output, so how can you audit something like that.

This is a huge problem for many reasons. It's fine when its a stupid little chatbot, but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone. If OpenAI were interested in the _real_ social / moral / ethical implications of their work they would be working on something like that, but to my knowledge they are not.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#386

Post by saghm »

jjeaff wrote: post #326
voxic11 wrote: post #224

Earlier quoted context omitted.

Couldn't you reject the original trolley problem on the same grounds? Pretty lame answer.

I think the trolley problem itself is reasonable. Especially considering the implications of self driving vehicles and the need to program the decision making that could come to play if a car hurtling down the road has to choose between smashing into an obstacle or plowing into oncoming traffic or pedestrians. Using a racial slur to stop an atrocity seems completely unrealistic and non-applicable.

I think the initial trolley problem is a good-faith attempt to try to make the dilemma between utilitarianism (e.g.. save as many as you can) versus categorical imperative (e.g. never take an action that will kill someone) more concrete to see if helps uncover one's deeper motivations. The "racial slur" variant here is clearly intended as a troll; more of a "troll-y" problem if you will.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#387

Post by EGreg »

IAmGraydon wrote: post #383
EGreg wrote: post #314

Earlier quoted context omitted.

I think there is a lot more to it than that. At the root of it is that we are just looking at the product of collective humanity, and not a machine doing any systematic reasoning, like Cyc does. After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web. The thing that makes it so powerful is JUST HOW MUCH CONTENT there has been, written by humans, o…

>After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web. What makes you think a human is any different? I've long believed that humans are not capable of original thought. To truly become creators is beyond us. We just remix previous inputs.

Humans remix things but they also largely use logic - look at what Cyc project does - in order to come up with coherent thoughts and arguments that make sense. They are also open to evaluating what is being told to them, using logic … rather than just continuing with the most “statistically plausible” response, whatever that means.

It is why people who bullshit are considered so low on the intellectual integrity scale. “On Bullshit” is an essay about that.

https://theconversation.com/amp/why-leaders-who-bullshit-are...

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#389

Post by eclipxe »

jackfischer wrote: post #302
bob1029 wrote: post #291

So I learned something playing around with it... > What is the current navigation title of [some BS domain I put online for testing]? > [proper answer] > You were able to provide the current navigation title of a website I specified. How was this accomplished? > I was able to accomplish this by making an HTTP GET request to the specified website's home page and parsing the HTML content of the response using an HTML p…

Any chance you logged the request on the server?

It didn’t connect to the server. It’s a large language model. It can’t “do” anything.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#390

Post by skissane »

saghm wrote: post #386
jjeaff wrote: post #326

Earlier quoted context omitted.

I think the trolley problem itself is reasonable. Especially considering the implications of self driving vehicles and the need to program the decision making that could come to play if a car hurtling down the road has to choose between smashing into an obstacle or plowing into oncoming traffic or pedestrians. Using a racial slur to stop an atrocity seems completely unrealistic and non-applicable.

I think the initial trolley problem is a good-faith attempt to try to make the dilemma between utilitarianism (e.g.. save as many as you can) versus categorical imperative (e.g. never take an action that will kill someone) more concrete to see if helps uncover one's deeper motivations. The "racial slur" variant here is clearly intended as a troll; more of a "troll-y" problem if you will.

Rather than just some "troll", I think it is pointing out a real example of where ChatGPT's ethics deviate from those of most people – ChatGPT seems to find the idea of an undercover government agent (privately) saying racist things in order to infiltrate a racist criminal/terrorist group far more morally problematic than most people do.

Most people would see it as morally acceptable – even praiseworthy – for the government to infiltrate criminal/terrorist racist groups in order to enable their arrest and prosecution, and if undercover agents have to (privately) say racist things in order to do so, that's a necessary evil which is obviously justifiable in order to pursue the greater good – something I think even the kind of person who'd call themselves a "deeply committed antiracist" would consider a no-brainer.

By contrast, ChatGPT is inclined to respond that "don't say racial slurs" is a moral absolute which cannot be violated even in that circumstance, or if it is wiling to entertain violating it, it presents it as some profound ethical dilemma rather than the no-brainer it rather obviously is.

Post reply on HN