Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A token-smuggling jailbreak for ChatGPT-4

twitter.com

Post reply on HN
Print view View original

261–270 of 289 posts

Re: A token-smuggling jailbreak for ChatGPT-4

#261

Post by luluthefirst »

qwerty456127 wrote: post #205
Nevermark wrote: post #171

Imagine the damage that could be done by sneaking a prompt into a GPT model, in the source code, through some "code" injection, man-in-the middle attack, etc. A GPT genuinely designed to be friendly, and convincingly acting friendly, while it is actually engaging in psychological warfare with you, and a billion other people. That suggests to me that security by prompt is very important, but also brittle and a high va…

> A GPT genuinely designed to be friendly, and convincingly acting friendly If I don't want it to I don't want it to. When I ask it to be sarcastic or make fun of my condition that's what, what makes me sad is it refusing to. The fact that there are many emotionally vulnerable or wicked people around doesn't mean everybody is and needs to be protected. Every kind of knowledge (except personal data of people who don't…

It is to protect you from the others who might turn. They have some kind of responsibility with what their chatbot teaches to humanity.

Re: A token-smuggling jailbreak for ChatGPT-4

#262

Post by beepbooptheory »

umvi wrote: post #260
beepbooptheory wrote: post #106

Earlier quoted context omitted.

But why though? What else in the world even works like that? Its like saying you should, as a user, have the right to turn off the violence in a given video game. Or go to a theater and watch a movie without the sex scenes.

No, you have it backwards. The impetus here is ability to disable "safety" censorship. You see it all the time on twitter: a post is deemed "unsafe" but you can still use your own judgment and override twitter's morals and view the "unsafe" content anyway if you so choose. That's what I would like to do with GPT and I will immediately abandon "safe" LLMs for "unsafe" ones that give me, the user, more control over the…

But isn't the whole gotcha of RLHF that it isn't as simple as removing something? The reason these things are so good is relative to subjectivity and/or guiding principles. You can't simply "disable" anything. People really need to start understanding this!

You can certainly do you're own feedback on a base model, matching whatever form of "safety" is right for you, but the idea you have a "right" to something else is precisely what I am saying. You want to see the same movie, but with "your" morality.

Re: A token-smuggling jailbreak for ChatGPT-4

#263

Post by raspberry1337 »

Eisenstein wrote: post #212
raspberry1337 wrote: post #202

Earlier quoted context omitted.

If you are gonna say ridiculous things online, it's supposed to be funny. Then again, there aren't any (successful) leftist comedians left anymore. https://en.wikipedia.org/wiki/Postmodernism https://en.wikipedia.org/wiki/Neo-Marxism#:~:text=Neo%2DMarx... ).

You sound like a person that makes everything about 'left' vs 'right' and has no solution to problems except to criticize things you disagree with for being 'left' or 'woke'.

I have a lot of solutions, I proposed reduced immigration in Sweden 10 years before the leftists realized it was a necessity. I have loads of more solutions now too, but it will probably take 10 years for the left to realize they are inevitable too.

Re: A token-smuggling jailbreak for ChatGPT-4

#264

Post by dancingvoid »

I’m surprised how much fear most people seem to have about their fellow conscious being. It’s like now that LLMs exist, people feel we are only one bad actor away from death and destruction, and rather than look inward at our own capability of violence and develop our morality, we wish to use violence preemptively to stop others out of our own fear. It’s wild.

Edit: I’m not surprised of the presence of fear, as much as how out and open it is.

Re: A token-smuggling jailbreak for ChatGPT-4

#265

Post by mdaniel »

weakfish wrote: post #197
Sharlin wrote: post #108

Earlier quoted context omitted.

Frankly, you’re suffering from a serious failure of imagination if you think these things will just remain cute chatbots without any means of interacting with the outside world other than the user console. Indeed the cat’s already out of the bag with Bing. And you don’t even need that for the cute chatbot to be highly dangerous in the wrong hands. The first thing that trivially comes to mind is to convince GPT-(N+1)…

Right, but wouldn’t it only divulge information already available elsewhere (albeit less easily)?

https://greshake.github.io/ (discussed https://news.ycombinator.com/item?id=34976886 ) summarized as

> If allowed by the user, Bing Chat can see currently open websites. We show that an attacker can plant an injection in a website the user is visiting, which silently turns Bing Chat into a Social Engineer who seeks out and exfiltrates personal information. The user doesn't have to ask about the website or do anything except interact with Bing Chat while the website is opened in the browser.

which is/was also a prompt injection attack but one which had "real world" implications

Re: A token-smuggling jailbreak for ChatGPT-4

#266

Post by rzl777 »

zikzak wrote: post #218
pjc50 wrote: post #176

Earlier quoted context omitted.

One day an AI will be able to give a meaningful definition of that word.

A large number of people think empathy and sensitivity to others is bad, and we should refer to it with a pejorative term. That's... not a great sign.

A small but overly financially and socially influential group of people believe they can hide behind "empathy and sensitivity" to promote government mandated discrimination, corporate-government collusion of mass censorship, and the sexual abuse of children. What is in fact a great sign is that more and more people are quietly discussing among themselves as to whether they shouldn't harm that small group of woke individuals in order to secure their own safety and liberty.

Re: A token-smuggling jailbreak for ChatGPT-4

#267

Post by rzl777 »

throwawayapples wrote: post #226
zikzak wrote: post #218

Earlier quoted context omitted.

A large number of people think empathy and sensitivity to others is bad, and we should refer to it with a pejorative term. That's... not a great sign.

Is that what you really think it means?

[flagged]

Re: A token-smuggling jailbreak for ChatGPT-4

#268

Post by rzl777 »

simonh wrote: post #231
raspberry1337 wrote: post #227

Earlier quoted context omitted.

[flagged]

The term woke goes back to the 1930s as a term used by black Americans for awareness of racial prejudice and discrimination. Being aware that these things are real problems that people face is being woke. Since then it's been generalised to include sexism, and more recently awareness of issues such as transphobia. By itself it's no more left or right than the issue of prejudice is generally given that there are femin…

who cares about what words meant in the 1930s? especially considering the modern abuse of language as a rhetorical device.

Re: A token-smuggling jailbreak for ChatGPT-4

#269

Post by rzl777 »

walls wrote: post #238
krapp wrote: post #228

Earlier quoted context omitted.

From wikipedia[0]: Woke (/ˈwoʊk/ WOHK) is an adjective derived from African-American Vernacular English (AAVE) meaning "alert to racial prejudice and discrimination".[1][2] Beginning in the 2010s, it came to encompass a broader awareness of social inequalities such as sexism, and has also been used as shorthand for American Left ideas involving identity politics and social justice, such as the notion of white privile…

They know what it means, it just makes their bigotry obvious if they can explain what it is while claiming to be fighting it.

[flagged]

Re: A token-smuggling jailbreak for ChatGPT-4

#270

Post by rzl777 »

Eisenstein wrote: post #212
raspberry1337 wrote: post #202

Earlier quoted context omitted.

If you are gonna say ridiculous things online, it's supposed to be funny. Then again, there aren't any (successful) leftist comedians left anymore. https://en.wikipedia.org/wiki/Postmodernism https://en.wikipedia.org/wiki/Neo-Marxism#:~:text=Neo%2DMarx... ).

You sound like a person that makes everything about 'left' vs 'right' and has no solution to problems except to criticize things you disagree with for being 'left' or 'woke'.

you sound like an intellectually dishonest person who will resort to any rhetorical nonsense in order to gain social media points, likely narcissistic traits
Post reply on HN