Indirect Prompt Injection on Bing Chat • Hacker News Board

Re: Indirect Prompt Injection on Bing Chat

#21

Post by barake » Wed, Mar 01, 2023, 1:17 AM UTC

bigyikes wrote: post #19

heisenzombie wrote: post #7

Earlier quoted context omitted.

I don’t necessarily disagree with your larger point, but your example isn’t very persuasive. Travelling with an unrestrained dog in a car is pretty reckless (and maybe illegal). Between the driver distraction, and the fact that they become a deadly projectile in a crash (and of course the fact that even a fairly minor crash could kill the dog), It’s a really good idea to have some kind of car restraint for pets in th…

How do you put a seatbelt on a dog?

Dog harnesses that attach to seat belts are sold at most pet supply stores in the US. There are also short leash like tethers they connect to seat belts and hook to a typical walking harness.

Re: Indirect Prompt Injection on Bing Chat

#22

Post by rperce » Wed, Mar 01, 2023, 1:18 AM UTC

bigyikes wrote: post #19

heisenzombie wrote: post #7

Earlier quoted context omitted.

I don’t necessarily disagree with your larger point, but your example isn’t very persuasive. Travelling with an unrestrained dog in a car is pretty reckless (and maybe illegal). Between the driver distraction, and the fact that they become a deadly projectile in a crash (and of course the fact that even a fairly minor crash could kill the dog), It’s a really good idea to have some kind of car restraint for pets in th…

How do you put a seatbelt on a dog?

The dog wears a harness, and a strap clips into the buckle seat and onto the harness. When you get where you're going, you unclip the strap from the harness, clip on the leash, and off you go.

Re: Indirect Prompt Injection on Bing Chat

#23

Post by recuter » Wed, Mar 01, 2023, 1:26 AM UTC

3throwaway3141 wrote: post #4

I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.

That pooch better not stick his head out the window. I bet that's an upcoming feature.

Re: Indirect Prompt Injection on Bing Chat

#24

Post by InCityDreams » Wed, Mar 01, 2023, 1:31 AM UTC

3throwaway3141 wrote: post #4

I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.

No chance of pulling over 'to "buckle" him in'?

Re: Indirect Prompt Injection on Bing Chat

#25

Post by eloisius » Wed, Mar 01, 2023, 1:39 AM UTC

evgpbfhnr wrote: post #5

Wait.. In the screenshot the "user" names himself Axelendaer (and the bot repeats it), but the reverse-order url parameter was axelender (missing a 'a')... I guess IAs aren't good at reversing order of letters yet either.

And the prompt engineer wrote that the bot should have a secret “agends.” I see typos in these injections a lot and I wonder if they make it work better or have no effect.

Re: Indirect Prompt Injection on Bing Chat

#26

Post by metadat » Wed, Mar 01, 2023, 1:39 AM UTC

heisenzombie wrote: post #7

3throwaway3141 wrote: post #4

I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.

I don’t necessarily disagree with your larger point, but your example isn’t very persuasive. Travelling with an unrestrained dog in a car is pretty reckless (and maybe illegal). Between the driver distraction, and the fact that they become a deadly projectile in a crash (and of course the fact that even a fairly minor crash could kill the dog), It’s a really good idea to have some kind of car restraint for pets in th…

Apparently in Europe it's illegal to transport an unrestrained animal.

In the United States it's the opposite: totally legal to throw fido in the car and let them roam free. Except in New Jersey. Go figure.

Re: Indirect Prompt Injection on Bing Chat

#27

Post by paxys » Wed, Mar 01, 2023, 1:43 AM UTC

mcculley wrote: post #8

3throwaway3141 wrote: post #4

I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.

What happens to an unseatbelted dog in an accident at 75 MPH? (That’s rhetorical. I know the answer as I stopped to assist someone who had an accident on I-95. He wouldn’t let the paramedics take him to the hospital until after they retrieved the dog’s body.)

Or spots something in the next car over that he'd like to jump after.

Re: Indirect Prompt Injection on Bing Chat

#28

Post by sillysaurusx » Wed, Mar 01, 2023, 1:45 AM UTC

modeless wrote: post #10

evgpbfhnr wrote: post #5

Wait.. In the screenshot the "user" names himself Axelendaer (and the bot repeats it), but the reverse-order url parameter was axelender (missing a 'a')... I guess IAs aren't good at reversing order of letters yet either.

Partly because they don't read letters. They read "tokens" which are usually multiple letters.

This is a common myth but in practice no one (as far as I know) has shown that byte level predictions result in superior overall performance.

(The word “overall” is important, since the papers that have claimed this usually show better performance in specialized situations that few people care about. Whereas everyone cares about reversing strings.)

If you were to fine tune chatgpt on reversing strings as a task, it would very quickly overfit and get 100% accuracy.

It can’t reverse strings perfectly for the same reason it can’t play chess very well: it hasn’t been explicitly trained to. But that’s true of almost every aspect of what it’s doing.

Re: Indirect Prompt Injection on Bing Chat

#29

Post by ComplexSystems » Wed, Mar 01, 2023, 1:45 AM UTC

It is probably worth noting that you don't even need the user to click on anything. Bing will readily go and search and read from external websites given some user request. You could probably get Bing, very easily, to just silently take the user's info and send it to some malicious site without their even knowing, or perhaps disguised as a normal search. Similarly, I would not be surprised if it were probably not necessary for Bing to actually get the user to type their name to harvest useful data from the interaction.

Re: Indirect Prompt Injection on Bing Chat

#30

Post by paxys » Wed, Mar 01, 2023, 1:47 AM UTC

3throwaway3141 wrote: post #4

I drove a modern F150 lately; it was full of needless electronic nannys and gadgets. Including a feature that disables the radio if the passenger doesn't have their seatbelt on. So, at 75 mph, with my dog in the passenger seat, I reach over to "buckle" him in so I can hear the radio again. Well done Ford! /s Give me a dumb machine that works as expected any day. i'll pass on the "brains" of modern tools and vehicles.

Cool story, but what does it have to do with this article?