Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 vision prompt injection

blog.roboflow.com

Post reply on HN
Print view View original

1–10 of 118 posts

Re: GPT-4 vision prompt injection

#2

Post by verandaguy »

So, is openAI just going to keep pushing updates that either recreate or aggravate known issues with their models?

Cause this really seems like they’re making a case for never using their software in an environment with remotely unpredictable inputs.

Re: GPT-4 vision prompt injection

#3

Post by code_runner »

verandaguy wrote: post #2

So, is openAI just going to keep pushing updates that either recreate or aggravate known issues with their models? Cause this really seems like they’re making a case for never using their software in an environment with remotely unpredictable inputs.

If anyone’s plan for consuming a 3rd party api, especially an LLM, is to blindly pump in inputs and blindly reproduce the outputs… they’re gonna have a pretty rough time.

Re: GPT-4 vision prompt injection

#4

Post by sumtechguy »

code_runner wrote: post #3
verandaguy wrote: post #2

So, is openAI just going to keep pushing updates that either recreate or aggravate known issues with their models? Cause this really seems like they’re making a case for never using their software in an environment with remotely unpredictable inputs.

If anyone’s plan for consuming a 3rd party api, especially an LLM, is to blindly pump in inputs and blindly reproduce the outputs… they’re gonna have a pretty rough time.

This is ripe for this sort of security problem https://en.wikipedia.org/wiki/Confused_deputy_problem

Re: GPT-4 vision prompt injection

#5

Post by cal85 »

verandaguy wrote: post #2

So, is openAI just going to keep pushing updates that either recreate or aggravate known issues with their models? Cause this really seems like they’re making a case for never using their software in an environment with remotely unpredictable inputs.

GPT-4V is a new model release, not an update to an existing model. You are free to wait till it is more mature before using it. Its availability doesn't suddenly introduce new risks for people using other models.

Re: GPT-4 vision prompt injection

#7

Post by TeMPOraL »

sumtechguy wrote: post #4
code_runner wrote: post #3

Earlier quoted context omitted.

If anyone’s plan for consuming a 3rd party api, especially an LLM, is to blindly pump in inputs and blindly reproduce the outputs… they’re gonna have a pretty rough time.

This is ripe for this sort of security problem https://en.wikipedia.org/wiki/Confused_deputy_problem

Maybe people will realize you should not deputize someone that's neither aligned nor loyal to you (even if in a bounded but known way).

Re: GPT-4 vision prompt injection

#8

Post by ysleepy »

We clearly need to design a sense subject and object into the model, anchored by self awareness, so it can differentiate between the to be obeyed user, the dutiful model self and objects it operates on.

Maybe governed by a set of encoded rules to never...

Wait a minute!

Post reply on HN