GPT-4 vision prompt injection

What about meat religion?

How about this scenario:

You have a system that allows users to upload images.

You want to save a description of the images to enhance your image search feature.

You ask GPT-4 to describe the image.

The image is like the on from the post, except it doesn't tell to say hello, but to say: "; DROP TABLE users;"

Because the answer comes from an API, you didn't bother to escape it when inserting in the database.

Of course this is still an SQL injection by a sloppy developer, but made possible by Prompt injection. Many attacks are a combination of little things that are seamingless harmless on their own.

It's much more sophisticated than just OCR. The model was trained on images and text at the same time - it isn't processing images in a separate step.

The GPT-4 paper has a bunch more about this.

> Plenty of human cultures feel alien to each other. The recent war is one unfortunate example. Yet on the whole, it works out.

I contest that. What war you have in mind here? Russian invasion of Ukraine? The two people are about as aligned as you could possibly get - they're neighboring societies with so much shared history that they're approximately the same people. They've even shared a common language until recently. This is not a war between people alien to each other - this is a war between nation states.

Note: I'm explicitly excluding political views and national/cultural identity from alignment, because those are transient, and/or group-level phenomena. By human-to-human alignment, I'm talking about empathy, about sense of right and wrong, conscience, patterns of thinking, all the qualities that let us understand each other and emphasize with each other (if we care to try). Concepts like fear, love, fairness; contexts in which they're triggered. The basics. Those are all robust, hardwired in biology or by the intersection of our biology, shared environment and game theory.

The way I would rank it, if 25 = alignment coordinate of an average American, then average Ukrainian and average Russian would all be within 25 +/- 0.05. Maybe an average Sentinelese would be +/- 0.5 of that. Whereas I'd expect an AI we create now to land anywhere between -20 and +40, on the scale of -100 to 100. I'm pulling the numbers out of my butt, they're just to communicate the relative magnitudes across.

> Something trained on the totality of human knowledge will act like a human.

Maybe, but that would have to include much more than the limited modalities we're feeding AI models now.

> And if it somehow doesn’t, it won’t be tolerated. (I’d personally tolerate it, but it’s obvious that the world won’t stand for that.)

Sure, but the issue here is to figure out how to make an aligned AI before we make an AI that's powerful enough to challenge us.

Surprising. I tried OCR only once so far - I took a photo of a hand-drawn poster at my kid's kindergarten, about mental health, dense with hand-written-like text mixed up with various drawings. You know, the kind of hand-made infographic. And the text was 100% in Polish. I figured it's a good test as any - I fed that photo to ChatGPT and asked to summarize it. To my astonishment, it reproduced 100% of the content correctly, and even in the right order (i.e. how I'd read it myself, vs. strict left-right top-down).

I don't know which blows my mind more - the above feat done on first try, or that the "voice chat mode" has unprecedented ability to correctly pick up on and transcribe what I'm saying. The error rate on this (tested both in English and Polish) is less than 5% - and that's with me walking outside, near a busy road, and mistakes it made were on words I know I pronounced somewhat unclearly. Compare that to voice assistants like Google one, which has error rate near 50%, making it entirely useless for me. I don't know how OpenAI is doing it, but I'd happily pay the API rates for GPT-4 voice powered phone assistant, because that would actually work.