91–100 of 182 posts
Wouldn't it be better to put "confetti" -- the API key as part of the domain name? That way, the key would be leaked without any required clicks due to the DNS prefetching by the browser.
Edit: Ah, wildcard subdomain? Does that get prefetched in Slack? Pretty terrible if so.
The key thing to understand here is the exfiltration vector. Slack can render Markdown links, where the URL is hidden behind the text of that link. In this case the attacker tricks Slack AI into showing a user a link that says something like "click here to reauthenticate" - the URL attached to that link goes to the attacker's server, with a query string that includes private information that was visible to Slack AI a…
It gets even worse when platforms blindly render img tags or the equivalent. Then no user interaction is required to exfil - just showing the image in the UI is enough.
Wouldn't it be better to put "confetti" -- the API key as part of the domain name? That way, the key would be leaked without any required clicks due to the DNS prefetching by the browser.
How would you own the server if you don't know what the domain is going to be? Perhaps I don't understand. Edit: Ah, wildcard subdomain? Does that get prefetched in Slack? Pretty terrible if so.
ie:
secret.attacker-domain.com will end up asking the dns for attacker-domain.com about secret.attacker-domain.com, and that dns server can log the secret and return an ip
Wouldn't it be better to put "confetti" -- the API key as part of the domain name? That way, the key would be leaked without any required clicks due to the DNS prefetching by the browser.
How would you own the server if you don't know what the domain is going to be? Perhaps I don't understand. Edit: Ah, wildcard subdomain? Does that get prefetched in Slack? Pretty terrible if so.
*.example.com. 14400 IN A 1.2.3.4
after that just collect webserver logs.
Earlier quoted context omitted.
This approach is flawed because it attempts to use use prompt-injection-susceptible models to detect prompt injection. It's not hard to imagine prompt injection attacks that would be effective against this prompt for example: https://github.com/gregretkowski/llmsec/blob/fb775c9a1e4a8d1... It also uses a list of SUS_WORDS that are defined in English, missing the potential for prompt injection attacks to use other lang…
Great, I would love to get some of the prompts you have in mind and try them with my library and see the results. Do you have recommendations on more effective alternatives to prevent prompt attacks? I don't believe we should just throw up our hands and do nothing. No solution will be perfect, but we should strive to a solution that's better than doing nothing.
Earlier quoted context omitted.
How would you own the server if you don't know what the domain is going to be? Perhaps I don't understand. Edit: Ah, wildcard subdomain? Does that get prefetched in Slack? Pretty terrible if so.
Wildcard dns would work: *.example.com. 14400 IN A 1.2.3.4 after that just collect webserver logs.
Earlier quoted context omitted.
The problem is that you cannot unteach it serving that shit. It's not like there is file you can delete. "It's a model, that's what it has learned..."
If you are implementing RAG - which you should be, because training or fine-tuning models to teach them new knowledge is actually very ineffective, then you absolutely can unteach them things - simply remove those documents from the RAG corpus.
Earlier quoted context omitted.
> I really don't understand why anyone expects LLM citations to be correct It can be done if you do something like: 1. Take user’s prompt, ask LLM to convert the prompt into a elastic search query (for example) 2. Use elastic search (or similar) to find sources that contain the keywords 3. Ask LLM to limit its response to information on that page 4. Insert the citations based on step 2 which you know are real sources…
Why would you expect step 3 to work?
Earlier quoted context omitted.
> Are companies really just YOLOing and plugging LLMs into everything Look we still can't get companies to bother with real security and now every marketing/sales department on the planet is selling C level members on "IT WILL LET YOU FIRE EVERYONE!" If you gave the same sales treatment to sticking a fork in a light socket the global power grid would go down overnight. "AI"/LLM's are the perfect shitstorm of just goo…
There's no global power grid. There are lots of local power grids.