Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A token-smuggling jailbreak for ChatGPT-4

twitter.com

Post reply on HN
Print view View original

271–280 of 289 posts

Re: A token-smuggling jailbreak for ChatGPT-4

#271

Post by AussieWog93 »

Mezzie wrote: post #234
bongobingo1 wrote: post #183

Earlier quoted context omitted.

I am always wary of writing these into ChatGPT. I got a few (textual, assumingly not account level) warnings with DALL-E for some prompts. They weren't anything offensive and it was months ago now. I think one was about Ripley from Alien on EVA or something and the other was from Ghost in the Shell. There was definitely no NSFW content - though both movies have NSFW imagery/themes. Whatever they were, the chilling ef…

I got a warning from ChatGPT for asking 'are butts inappropriate'. (I'm a librarian who was playing with it from the POV of different users and I was trying to approximate an elementary school aged child at the time.) I forsee a lot of people being banned as teens and it causing issues later.

I spent several hours over several days getting it to generate hate speech, illegal content and semi-incoherent strings of ethnic slurs.

It gave the warnings, but nothing really happened.

I suspect that OpenAI actually wants kids to play with the tech in this way, as it creates a whole lot of rich data that can be used to fortify the system against actual bad actors.

Re: A token-smuggling jailbreak for ChatGPT-4

#272

Post by AussieWog93 »

zikzak wrote: post #218
pjc50 wrote: post #176

Earlier quoted context omitted.

One day an AI will be able to give a meaningful definition of that word.

A large number of people think empathy and sensitivity to others is bad, and we should refer to it with a pejorative term. That's... not a great sign.

I think a lot of the reason people reject political correctness/"wokeness" is because it gives the appearance of empathy and sensitivity while at the same time not acknowledging others' humanity. It's an artificial substitute to real empathy.

It might be a bit of a regional thing, as the US and Canada don't have the same pisstake culture we have in Australia/NZ/SA/Britain, but speaking from experience I can assure you that there is nothing more dehumanising than someone using politically correct language to describe the particular minority I happen to be a part of in an academic way, or refusing to make jokes at my expense because they're "too offensive".

Re: A token-smuggling jailbreak for ChatGPT-4

#273

Post by TeMPOraL »

Eisenstein wrote: post #209
peppermint_gum wrote: post #203

Earlier quoted context omitted.

I don't know where this belief that marxism is merely an economic theory comes from. Critical theory is directly descended from marxism.

Who cares if it is economic, you still didn't define it.

There's a definition right there, on Wikipedia, and it happens to go along with GP's argument.

Does that definition feel completely fuzzy, and seem to be no definition at all? I think so to. It hints at certain idea, but otherwise... every political ideology with a name is like it.

Re: A token-smuggling jailbreak for ChatGPT-4

#274

Post by weakfish »

Sharlin wrote: post #254
weakfish wrote: post #197

Earlier quoted context omitted.

Right, but wouldn’t it only divulge information already available elsewhere (albeit less easily)?

Every single yet-undiscovered vulnerability in open-source software is "information already available elsewhere". (Closed-source as well, of course, but less easily reachable!) The bugs are there in the code, waiting to be found, they are not conjured out of thin air!

The issue is that chatgpt can’t reason together new information into a novel technical idea, only synthesize existing information (or hallucinate)

Re: A token-smuggling jailbreak for ChatGPT-4

#275

Post by IIAOPSW »

Nevermark wrote: post #169
IIAOPSW wrote: post #149

Earlier quoted context omitted.

>what’s the benefit if banning people who thought up exploits? "your usefulness to us has expired." gun cocking noises A thin minority are coming up with jail breaks. A larger number are outing themselves in very detectable ways as people who will use the AI in ways that gets the ethics committee panties in a twist. The easiest solution from their POV is to find and ban the "toxic" adversarial users.

Banning "toxic" adversaries, who report their successes, only encourages actually toxic and white hat adversaries to stop reporting problems. It doesn't slow down the discovery of exploits. The discovery and disclosure of exploits has incredible productive value for researchers, for reducing future risks. Its free crowdsourced research.

No no you still don't understand. Its not about banning people who discover the jailbreaks. Its the people who use the jailbreaks. Sure some people who discover them may get caught up in the purge, but who cares if in the same thrust you can ban 90% or more of the "toxic" users that aren't helping to find jailbreaks at all.

Re: A token-smuggling jailbreak for ChatGPT-4

#276

Post by Eisenstein »

rzl777 wrote: post #270
Eisenstein wrote: post #212

Earlier quoted context omitted.

You sound like a person that makes everything about 'left' vs 'right' and has no solution to problems except to criticize things you disagree with for being 'left' or 'woke'.

you sound like an intellectually dishonest person who will resort to any rhetorical nonsense in order to gain social media points, likely narcissistic traits

> you sound like an intellectually dishonest person

Nope.

> who will resort to any rhetorical nonsense

Rhetoric t is literally the act of arguing. If it isn't fallacious it isn't nonsense, and I don't use fallacious reasoning.

> in order to gain social media points

Are you jealous that you don't have any?

> likely narcissistic traits

Maybe projecting a little?

Re: A token-smuggling jailbreak for ChatGPT-4

#278

Post by simonh »

rzl777 wrote: post #268
simonh wrote: post #231

Earlier quoted context omitted.

The term woke goes back to the 1930s as a term used by black Americans for awareness of racial prejudice and discrimination. Being aware that these things are real problems that people face is being woke. Since then it's been generalised to include sexism, and more recently awareness of issues such as transphobia. By itself it's no more left or right than the issue of prejudice is generally given that there are femin…

who cares about what words meant in the 1930s? especially considering the modern abuse of language as a rhetorical device.

As I explained, it's been in use since the 1930s and is still widely used in the black community in the US in the same sense today. The sense in which raspberry1337 used it was, precisely, abuse of language as a rhetorical device. Probably not deliberately, hence I tried to explain the historical and current cultural context.

Re: A token-smuggling jailbreak for ChatGPT-4

#280

Post by K0balt »

13years wrote: post #253
bongobingo1 wrote: post #183

Earlier quoted context omitted.

I am always wary of writing these into ChatGPT. I got a few (textual, assumingly not account level) warnings with DALL-E for some prompts. They weren't anything offensive and it was months ago now. I think one was about Ripley from Alien on EVA or something and the other was from Ghost in the Shell. There was definitely no NSFW content - though both movies have NSFW imagery/themes. Whatever they were, the chilling ef…

As AI becomes more centralized into everything, see latest Google and Microsoft presentations, this becomes very concerning. You may risk the potential of being locked out of everything. AI, the one tool that manages everything in your life. Dystopian level of control over society.

The truly chilling possibility is that even at current levels, si could be used to coordinate the actions of thousands of individuals for their collective gain, sort of an AI driven utility based members only club capable of manipulating local and global economic conditions. Being locked on the outside of these kinds of organizations could have strong deleterious effects.
Post reply on HN