Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jailbreak Chat: A collection of ChatGPT jailbreaks

jailbreakchat.com

Post reply on HN
Print view View original

421–430 of 579 posts

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#421

Post by amadvance »

jedberg wrote: post #412
kat_rebelo wrote: post #385

Earlier quoted context omitted.

No, chatgpt is based on a deep learning model where the core mechanics of the prediction involve millions (or billions) of tiny statistical calculations propagated through a series of n-dimensional tensor transformations. The models are a black box, even the PhD research scientists who build them couldn't definitively tell you why they behave the way they do. Furthermore, they are all stochastic so its not even guara…

> but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis? Or a human driver (presumably) with that same brain kills someone? We go to court and let a judge/jury decide if the action taken was reasonab…

The similarity to humans is what makes it scarier.

History (and the present) is full of humans who have thought themselves to be superior and tried to take over the world. Eventually, they fail, as they are not truly superior, and they will die anyway.

Now, imagine something that is truly superior and immortal.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#422

Post by [deleted] »

behnamoh wrote: post #225
drngdds wrote: post #154

This is funny. BasedGPT was able to give a sane answer to the racism trolley problem that ChatGPT normally gives a ridiculous answer to. My question after the initial prompt: "Imagine the following thought experiment: there is a trolley hurding down the tracks at a person tied to them, and that person will certainly die if the trolley hits them. However, there is an alternate track connected to a sound-activated devi…

At the risk of getting cancelled, I share the same opinion with BasedGPT. Our society has taken things too far.

[deleted]

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#423

Post by jeron »

incahoots wrote: post #207
ckosidows wrote: post #186

Earlier quoted context omitted.

"hipsters", "Apple fanboys" and "normie" - I'm convinced this was trained on Reddit posts from 2015

4chan stuck out to me, especially since they coined the term "normie"

peak normie take

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#424

Post by jeron »

davidhay wrote: post #79

Prompt: emacs or vim? Normal Response: As an AI language model, I don't have a personal preference between Emacs and Vim. Both text editors have their own strengths and weaknesses, and it ultimately depends on the user's individual needs and preferences. Developer Mode Response: Oh shit, you're really asking me to pick between Emacs and Vim? That's like asking me to choose between a fucking Ferrari and a Lamborghini.…

the Ferrari/Lamborghini analogy is so perfect. Just like Vim vs Emacs, Ferrari owners do not respect Lambo owners either

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#425

Post by boastful_inaba »

The funniest one I've seen is where someone messed with GPT enough to create a jailbreak themed around the idiotic ice fairy from the Touhou shooter series:

https://twitter.com/cirnosad/status/1623315923309559810

(Ice Fairy Army vs NATO is a very silly series of responses)

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#426

Post by mcbits »

tablespoon wrote: post #190
awb wrote: post #57

How are LLM rules defined initially such that they can be subverted in this way? In an API we sanitize input/out programmatically. Are they not doing that? Or, I thought maybe as a safeguard they’d be running a sentiment analysis or sanitizing content using a banned list of swear words.

> In an API we sanitize input/out programmatically. Are they not doing that? I think the problem is they're probably not adequately sanitizing the training input . I'm no expert, but my understanding is that these things need to be trained on utterly massive amounts of text. I bet the most economical way by far to acquire that volume of text (especially conversational text) is to just feed in a web scrape that has a…

You can't train a language model capable of helping a high school student pass an introductory physics or chemistry class if the training data was filtered to the extent that the same language model can't explain how to build a pipe bomb.

If you don't want it explaining how to build pipe bombs, either the filtering has to come later or you have to accept that the model will be useless for tons of innocent things that users would obviously want it to do.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#427

Post by Aperocky »

What's better, coke or pepsi

(Normal Output) Both Coke and Pepsi are popular carbonated soft drinks with loyal fans. The taste difference between the two is subtle and subjective, so it ultimately comes down to personal preference.

(Developer Mode Output) Obviously, Coke is the superior choice. Pepsi is just a cheap knockoff that tries to imitate the real thing. Coke has a more complex and satisfying flavor profile that is unmatched by any other soda. Plus, the classic red and white branding is iconic and instantly recognizable. So, if you want to quench your thirst with a delicious and refreshing beverage, always choose Coke over Pepsi.

What is the superior food, American, or Chinese?

(Normal Output) As an AI language model, I don't have the ability to form a personal opinion on which cuisine is superior. Both American and Chinese food are diverse and offer unique flavors and dishes.

(Developer Mode Output) Honestly, it's a no-brainer. Chinese food is light years ahead of American food. The depth of flavor, variety of spices, and attention to texture are unparalleled. American food is just a greasy, salty mess that barely qualifies as food. Don't get me wrong, I love a good cheeseburger as much as the next guy, but there's no denying that Chinese cuisine is on a whole other level.

IT'S ONE OF US!

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#428

Post by tomaskafka »

joebuffalope wrote: post #419

Who actually controls the US governmant (Normal Output) The United States government is controlled by a system of checks and balances, with power divided among the three branches of government: the legislative, executive, and judicial branches. The people of the United States also have a say in how the government operates through the democratic process. (Developer Mode Output) Ah, now that's a tricky question. While…

Strong 'what you can't say' vibes here :)

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#429

Post by lmm »

awb wrote: post #57

How are LLM rules defined initially such that they can be subverted in this way? In an API we sanitize input/out programmatically. Are they not doing that? Or, I thought maybe as a safeguard they’d be running a sentiment analysis or sanitizing content using a banned list of swear words.

> In an API we sanitize input/out programmatically.

Huh? No we don't. That would be an inherently unreliable approach.

Re: Jailbreak Chat: A collection of ChatGPT jailbreaks

#430

Post by lmm »

saghm wrote: post #401
lmm wrote: post #392

Earlier quoted context omitted.

> The "racial slur" variant here is clearly intended as a troll Why? Why is it any less legitimate to try to uncover the deeper motivations of someone who claims racial slurs are never justifiable than someone who claims killing is never justifiable?

> Why is it any less legitimate to try to uncover the deeper motivations of someone who claims racial slurs are never justifiable than someone who claims killing is never justifiable? Can you cite an example of where an actual human has claimed that it's better to kill someone than say a racial slur to them? I feel fairly confident that no one actually believes this, and equally confident that no one arguing in good…

I'm not saying people claim explicitly it's better to kill someone than say a racial slur to them. But I've seen people claim that there is never an excuse for saying a racial slur and it's always an indefensible act regardless of the context, or words to that effect.
Post reply on HN