I predict prompt injection will be a common vulnerability for years to come since the natural language advantages of LLMs is the same advantage hackers will have
Ask HN: Is “prompt injection” going to be a new common vulnerability?
111–116 of 116 posts
Re: Ask HN: Is “prompt injection” going to be a new common vulnerability?
#112Worrying about the retrieving unwanted info from the models pales in comparison to the issues we'll have when people start hooking these things up to external systems. It matters a lot more when your WhateverGPT hallucinates if it has access to the rest of the computer or parts of the physical work.
Re: Ask HN: Is “prompt injection” going to be a new common vulnerability?
#113Yes. Prompt injection will continue to be a common vulnerability for quite a while, from what I've seen. I wrote a bunch about this back in September: - https://simonwillison.net/2022/Sep/12/prompt-injection/ was I believe the first blog entry to use the term "prompt injection" - https://simonwillison.net/2022/Sep/16/prompt-injection-solut... - "I don't know how to solve prompt injection" - talks about how, unlike at…
“You can’t solve programming security problems with more programming.” I think it’s definitely possible to detect “escape” attempts, and to train the model in the first place to respect its directions. It’s just not an actual security problem, the models contain no secrets nor control any levers. You get some bad optics is all.
So far
Re: Ask HN: Is “prompt injection” going to be a new common vulnerability?
#114It's fascinating how risky these systems will become if they're deployed anywhere sensitive
Re: Ask HN: Is “prompt injection” going to be a new common vulnerability?
#115Earlier quoted context omitted.
> the danger seems to be that there is no currently documented way to completely remove these possible outputs I know OpenAI likes throwing around terms like "danger" and "harm" liberally, but is this really a danger ? Outside of hypothetical scenarios where someone wires ChatGPT to a self-driving trolley.
Yes, there are absolutely dangers. It shouldn't be possible for a depressed person to convince a chatbot to tell them to commit suicide. There are some people who only need the tiniest push on a bad day.
by that logic. every single song or work of art should go through a filtration process before being distributed.
we shouldn't ruin the capabilities of a tool just because some minority of people would use it to justify undesirable actions.
Re: Ask HN: Is “prompt injection” going to be a new common vulnerability?
#116Earlier quoted context omitted.
>""Open""AI OpenAI was never open. It's named that to invoke good feelings, not because there's any meaningful 'openness' to their work.
They released CLIP and they release papers/descriptions of their work for all their other products, enough for people to reproduce them. That's quite open.
Apple, nVidia, Microsoft, Intel, ... release plenty of papers and we don't call them open.