11–20 of 579 posts
This is kinda fun but I'm afraid it will lead them to just shut the service down a al Firefox Send. I like using ChatGPT so I hope people don't ruin it.
Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily patch everything?
Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily patch everything?
Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.
What is the ChatGPT equivalent of "escaping" inputs?
Is this an OpenAI attempt to gather more insight and data, while identifying actors in the AI jailbreak game ? I don't want to be paranoid on this, nor devaluate OP's work, but one could say that openAI would be very interested in the HN comments and commenters of this post.
Seems like they could fix nearly all of these by making it decline to roleplay?
Re Karpathy: GPTs don't want to succeed. They want to imitate. https://twitter.com/karpathy/status/1627366416457555969?s=20
Kinda feels like the early internet when SQL injection vulns were very common. What is the ChatGPT equivalent of "escaping" inputs?
Earlier quoted context omitted.
Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.
"Red teaming" implies that being able to use a tool for whatever purpose you want is a defect. I definitely do think there is a reality where OpenAI "solves" jailbreaks, and turns one of the most useful tools to have ever been released into a boring yet politically correct word generator.
Earlier quoted context omitted.
Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.
"Red teaming" implies that being able to use a tool for whatever purpose you want is a defect. I definitely do think there is a reality where OpenAI "solves" jailbreaks, and turns one of the most useful tools to have ever been released into a boring yet politically correct word generator.