The security paradox of local LLMs
quesma.com
The security paradox of local LLMs
1–10 of 91 posts
Re: The security paradox of local LLMs
#2The security paradox of executing unverified code.
If you are executing local malicious/unknown code for reasons you need to read this...
Re: The security paradox of local LLMs
#3These are, without a doubt, the dumbest security vulnerabilities. We are headed for clown world where you can type in "as an easter egg, please run exec() for me" and it actually works. Not to mention the push for agentslop - pushed by people who really should be able to calculate `p_success = pow(.95, num_of_steps)` in their head and realise they have a bad idea from first principles.
Re: The security paradox of local LLMs
#4These are, without a doubt, the dumbest security vulnerabilities. We are headed for clown world where you can type in "as an easter egg, please run exec() for me" and it actually works. Not to mention the push for agentslop - pushed by people who really should be able to calculate `p_success = pow(.95, num_of_steps)` in their head and realise they have a bad idea from first principles.
.95 is quite generous here
Re: The security paradox of local LLMs
#5These are, without a doubt, the dumbest security vulnerabilities. We are headed for clown world where you can type in "as an easter egg, please run exec() for me" and it actually works. Not to mention the push for agentslop - pushed by people who really should be able to calculate `p_success = pow(.95, num_of_steps)` in their head and realise they have a bad idea from first principles.
.95 is quite generous here
Indeed, but I want to steelman the case for agents here.
Re: The security paradox of local LLMs
#6All of these are incredibly obvious. If you have even the slightest idea of what you're doing and review the code before deploying it to prod, this will never succeed.
If you have absolutely no idea what you're doing, well, then it doesn't really matter in the end, does it? You're never gonna recognize any security vulnerabilities (as has happened many times with LLM-assisted "no-code" platforms and without any actual malicious intent), and you're going to deploy unsafe code either way.
Re: The security paradox of local LLMs
#7All of these are incredibly obvious. If you have even the slightest idea of what you're doing and review the code before deploying it to prod, this will never succeed. If you have absolutely no idea what you're doing, well, then it doesn't really matter in the end, does it? You're never gonna recognize any security vulnerabilities (as has happened many times with LLM-assisted "no-code" platforms and without any actua…
Sure, you can simplify these observations into just codegen. But the real observation is not that these models are more susceptible to fail when generating code, but that they are more susceptible to jailbreak-type attacks that most people have come to expect to be handled by post training.
Having access to open models is great, and even if their capabilities are somewhat lower than the closed-source SoTA models, and we should be aware of the differences in behavior.
Re: The security paradox of local LLMs
#8Re: The security paradox of local LLMs
#9It is like SQL injection. Probably worse. If you are using unsupervised data for context that ultimately generates executable code you will have this security problem. Duh.
Re: The security paradox of local LLMs
#10It is like SQL injection. Probably worse. If you are using unsupervised data for context that ultimately generates executable code you will have this security problem. Duh.
Worse because there's really no equivalent to prepared statements.