321–330 of 419 posts
Earlier quoted context omitted.
> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.
For a "modern" programmer a .sh file hosted in some random webserver which you tell him to wget and run would be best.
I wish you were wrong.
Earlier quoted context omitted.
> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.
> an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day I think the truly average white collar worker more or less blindly clicks anything and everything if they think it will make their work/life easier...
Earlier quoted context omitted.
> you cannot prevent prompt injection I wonder if might be possible by introducing a concept of "authority". Tokens are mapped to vectors in an embedding space, so one of the dimensions of that space could be reserved to represent authority. For the system prompt, the authority value could be clamped to maximum (+1). For text directly from the user or files with important instructions, the authority value could be cl…
The model only sees a stream of tokens, right? So how do you signal a change in authority (i.e. mark the transition between system and user prompt)? Because a stream of tokens inherently has no out-of-band signaling mechanism, you have to encode changes of authority in-band . And since the user can enter whatever they like in that band... But maybe someone with a deeper understanding can describe how I'm wrong.
Earlier quoted context omitted.
Yea agreed. Or use RBAC
RBAC doesn't help. Prompt injection is when someone who is authorized causes the LLM to access external data that's needed for their query, and that external data contains something intended to provoke a response from the LLM. Even if you prevent the LLM from accessing external data - e.g. no web requests - it doesn't stop an authorized user, who may not understand the risks, from pasting or uploading some external d…
Earlier quoted context omitted.
> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.
> an average white-collar workers will find .md much more suspicious *.dmg files on macOS are even worse! For years I thought they'd "damage" my system...
Well, would you argue that the office apps you installed from them didn't cause you damage, physically or emotionally?
One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…
In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more succes…
You can even add a nice "copy to clipboard button" that copies something entirely different than what is shown, but it's unnecessary, and people who are more careful won't click that.
They’re passing in half the internet via rag and presumably didn’t run a llamaguard type thing over literally everything?
Earlier quoted context omitted.
it is less of a problem for revoking attacker's keys (but maybe it has access to victim's contents?). agreed it shouldn't be used to revoke non-malicious/your own keys
The poster you originally replied to is suggesting this for revoking the attackers keys. Not for revocation of their own keys…