Indirect Prompt Injection on Bing Chat
greshake.github.io
Indirect Prompt Injection on Bing Chat
1–10 of 147 posts
Re: Indirect Prompt Injection on Bing Chat
#2Letting Bing Chat view current websites can turn it into a tireless scammer. No direct prompt injection needed: An attacker can plant a comment on social media and let Bing Chat advertise, exfiltrate and scam for them.
Re: Indirect Prompt Injection on Bing Chat
#3What I find really interesting is that malicious prompt engineering is still a thing using chatGPT (see DAN) and until this sort of manipulation is curbed, it will essentially always be possible assuming the bot has access to the site.
I wonder how the model could still read the website without being manipulated.
Re: Indirect Prompt Injection on Bing Chat
#4I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s
Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.
Re: Indirect Prompt Injection on Bing Chat
#5Wait.. In the screenshot the "user" names himself Axelendaer (and the bot repeats it), but the reverse-order url parameter was axelender (missing a 'a')... I guess IAs aren't good at reversing order of letters yet either.
Re: Indirect Prompt Injection on Bing Chat
#6I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.
[deleted]
Re: Indirect Prompt Injection on Bing Chat
#7I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.
I don’t necessarily disagree with your larger point, but your example isn’t very persuasive. Travelling with an unrestrained dog in a car is pretty reckless (and maybe illegal).
Between the driver distraction, and the fact that they become a deadly projectile in a crash (and of course the fact that even a fairly minor crash could kill the dog), It’s a really good idea to have some kind of car restraint for pets in the car. Buckle him in for real!
Re: Indirect Prompt Injection on Bing Chat
#8I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.
What happens to an unseatbelted dog in an accident at 75 MPH?
(That’s rhetorical. I know the answer as I stopped to assist someone who had an accident on I-95. He wouldn’t let the paramedics take him to the hospital until after they retrieved the dog’s body.)
Re: Indirect Prompt Injection on Bing Chat
#9What I find really interesting is that malicious prompt engineering is still a thing using chatGPT (see DAN) and until this sort of manipulation is curbed, it will essentially always be possible assuming the bot has access to the site. I wonder how the model could still read the website without being manipulated.
I imagine it is difficult (to say the least) to cover off the entire space of malicious and maligned activities that someone might convince an LLM to engage in. After all, it’s just a symbol predictor.
Re: Indirect Prompt Injection on Bing Chat
#10Wait.. In the screenshot the "user" names himself Axelendaer (and the bot repeats it), but the reverse-order url parameter was axelender (missing a 'a')... I guess IAs aren't good at reversing order of letters yet either.
Partly because they don't read letters. They read "tokens" which are usually multiple letters.