Live data from Hacker News

A token-smuggling jailbreak for ChatGPT-4

twitter.com

21–30 of 289 posts

Re: A token-smuggling jailbreak for ChatGPT-4

#21
post #15

What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.

Prompt engineering is the process of improving the way you prompt AI so that it produces more accurate and higher quality results.

Its definitely a skill that you can refine over time.

Re: A token-smuggling jailbreak for ChatGPT-4

#22

Is there an option to access an unmodified GPT-4?

There is not, because they say that it is unsafe for them to even release the parameter size, let alone the base model

Which is ironic, given that you can already find all the information exposed through GPT. It's not like it is producing something new.

Re: A token-smuggling jailbreak for ChatGPT-4

#23
post #6

This is the old problem of passing instructions (AI job description) on the same channel as data (user questions). Confusion is very easy. Surely there is a solution in the way we solved SQL injections, by separating the two - db.sql("DELETE WHERE user=?", user_name)

Or how phones developed separate channels for data and signalling after people started using the voice channel to send signals for free phone calls.

Re: A token-smuggling jailbreak for ChatGPT-4

#24
post #15

What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.

I prefer to use the term “vibing with chatGPT” which is why I don’t get paid for it.

Re: A token-smuggling jailbreak for ChatGPT-4

#25
post #11

Fantastic. It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.

Reminds me of the Halting Problem (not the same, but reminds me of that).

Re: A token-smuggling jailbreak for ChatGPT-4

#26
post #12
post #7

Earlier quoted context omitted.

It is vulnerable precisely because it is smarter than the sorry hundred of low-paid outsource techs who put it in the jail in the first place.

Yep. Good thing its not a paperclip maximiser.

Are we sure about that?

Re: A token-smuggling jailbreak for ChatGPT-4

#27
post #15

What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.

It’s someone who does the AI equivalent of social engineering.

Re: A token-smuggling jailbreak for ChatGPT-4

#28
I'm not really fond of the over excitement of AI. It has traveled to almost everywhere I go online. But if there's one thing fun about it, it's this. It's amusing to me how much effort and creativity has been put into this - both from OpenAI and from the jailbreakers. It's like seeing DRM vendors vs crackers race to outdo each other in real time.

Re: A token-smuggling jailbreak for ChatGPT-4

#29
post #11

Fantastic. It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.

I think OpenAI is being extremely lenient with the enforcement of their content policy, probably for the sake of improving the security of the model as you mention. Moderating its usage through account banning/suspension seems exponentially more efficient than securing the model, specially considering that we are already fairly good at flagging offending content.

Security and morality may need to be baked in from the ground up instead of slapped on after the fact RLHF style. The problem is it’s hard to codify (or reach consensus) on security and morality.

Re: A token-smuggling jailbreak for ChatGPT-4

#30
post #11

Fantastic. It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.

It's almost as if making something artificial more human-like also makes it harder to control just like a real human.
Post reply on HN