Viewing profile — greshake

(depending on whether the search engine context is connected to untrusted inputs or only your curated database ofc)

You're missing the more important vector for prompt injection: Indirect injection through the "search engine context". It's not just a matter of blocking bad user questions to fend…

Check out my blog where I show even more up-to-date techniques and the insane ways vulnerable applications are being deployed: https://kai-greshake.de/ Here I go through all of the…

TLDR: With these vulnerabilities, we show the following is possible: - Remote control of chat LLMs - Persistent compromise across sessions - Spread injections to other LLMs - Compr…

Neither is possible right now.

I just published a blog post showing that that is not what is happening. Companies are plugging LLMs into absolutely anything, including defense/threat intelligence/cybersecurity/l…

I've written about this extensively. My latest article goes into the consequences. How about going from Prompt Injection to airstrike? https://kai-greshake.de/posts/in-escalating-o…

Also check out a blogpost on the same subject: https://kai-greshake.de/posts/llm-malware/

"We demonstrate our attacks' practical viability against both real-world systems, such as Bing's GPT-4 powered Chat and code-completion engines, and synthetic applications built on…

Look into our repo (also linked there) we started out with only demonstrating that it works on GPT-3 APIs, now we also know it works on ChatGPT/3.5-turbo with ChatML and GPT-4, and…

> your finding that ChatML does not work seems to be mainly this Also the fact that ChatML has been broken into bits many, many times now- see again the prompt golfing. Also I'm ta…

Segmenting different data sources is the main approach pursued by OpenAI afaik (ChatML for example). That has not worked so far, as you can see in this prompt golfing game: https:/…

[...] demonstrate potentially brutal consequences of giving LLMs like ChatGPT interfaces to other applications. We propose newly enabled attack vectors and techniques and provide d…

Soo.. Expect your personal GPT to be persistently compromised/hacked, remote-controlled and used to exfiltrate all your data. Security of LLMs is in a bad state right now.