Isn't this the dream? Let marketing ship and own their features while swe do the engineering?
The VibeSec Reckoning
21–28 of 28 posts
Re: The VibeSec Reckoning
#22First so called vulnerability, isn't how a lot platforms are actually built? Share a link/copy a link, and more often than not, I am sure to have read a warning like "anyone with that link may access that file". Now should I mention all the screw up I have seen in several Saas 1b+ valuation, including DocuSign/ and more security oriented ones (PIM related etc?). For any softwares, you need a minimum critical mindset…
Well it depends on the url. Usually shareable url where "anyone with the link may access that file" contain a random element that makes it hard to guess if you don't have it (e.g. an UUID).
In other cases the content is at easily guessable path, and that is a whole different story
Re: The VibeSec Reckoning
#23Every issue with AI output lies somewhere on a continuum of how challenging it is to mitigate with pure model training.
Syntax is solved.
But getting agents to write secure code currently seems beyond what can be trained into models even with synthetic data. Or maybe the big labs haven’t tried yet.
Regardless if you truly care about your AI’s output having some property, the only way is to codify how work that has that quality looks - then create deterministic hooks and checks that refuse to let the AI stop until it’s passed the bar.
Skills, MCPs, “you MUST do this” in your agent instructions.. it’s all just new ways to waste tokens trying to asymptotically approach what good work looks like.
You will never reliably get acceptable work unless you build deterministic checking, and enforcement of said checking in a way to model can’t bypass or ignore.
Look into Claude Code hooks - your hook can be a script, and if it exits with exit code 2, it’ll block the model and show it the script’s output. A stop hook can check the model’s work and block its attempt to stop if the work doesn’t meet your bar. The script output can describe what still needs to be fixed and for bonus points, where (line 567 uses untrusted input, paragraph 2 makes an uncited claim, clause 15 references superseded case law, etc.)
Re: The VibeSec Reckoning
#24Something worth noting is that the types of vulnerabilities LLMs introduce are notably different from what humans introduce, way fewer local issues like syntax mistakes, simple memory problems, etc and far more broad issues like authn/authz
What do you mean about syntax mistakes and memory problems?
Something like incorrect SELinux configurations?
Re: The VibeSec Reckoning
#25Every issue with AI output lies somewhere on a continuum of how challenging it is to mitigate with pure model training. Syntax is solved. But getting agents to write secure code currently seems beyond what can be trained into models even with synthetic data. Or maybe the big labs haven’t tried yet. Regardless if you truly care about your AI’s output having some property, the only way is to codify how work that has th…
Anecdotally, a few weeks into a Rust agent-first project, we're still trying to get the agent to maintain a minimum of coding discipline (e.g. don't use sync Mutex in tokio code). So far, the agent seems more interested in deactivating the linters than in complying.
Security? At this stage, I'm a bit afraid that it's a joke more than anything else.
Re: The VibeSec Reckoning
#26Every issue with AI output lies somewhere on a continuum of how challenging it is to mitigate with pure model training. Syntax is solved. But getting agents to write secure code currently seems beyond what can be trained into models even with synthetic data. Or maybe the big labs haven’t tried yet. Regardless if you truly care about your AI’s output having some property, the only way is to codify how work that has th…
Anecdotally, a few weeks into a Rust agent-first project, we're still trying to get the agent to maintain a minimum of coding discipline (e.g. don't use sync Mutex in tokio code). So far, the agent seems more interested in deactivating the linters than in complying. Security? At this stage, I'm a bit afraid that it's a joke more than anything else.
That should be solvable by denying permission to edit the lint files with a message saying lint files cannot be edited and not to use workarounds (sed, scripting etc)
You could also use hooks to block running of scripts for some number of turns after an attempt to cheat.
Re: The VibeSec Reckoning
#27Vibe coding into production? You don't need to wait for scientists to produce research to know that's not a great idea. You played yaself
We do it, it’s fine. From what I can tell just about every company does it now. Review your code, have integration tests, rollout feature incrementally with feature flags. All the things we previously did for all the really bad human developers which AI is way better than
> All the things we previously did for all the really bad human developers which AI is way better than
They defined vibecoding as non technical folk using llms to code, sound like you were a software house and have coding experience... Basically id trust coders with prod released ai code, but not an office full of Jackies
Re: The VibeSec Reckoning
#28Earlier quoted context omitted.
Anecdotally, a few weeks into a Rust agent-first project, we're still trying to get the agent to maintain a minimum of coding discipline (e.g. don't use sync Mutex in tokio code). So far, the agent seems more interested in deactivating the linters than in complying. Security? At this stage, I'm a bit afraid that it's a joke more than anything else.
That should be solvable by denying permission to edit the lint files with a message saying lint files cannot be edited and not to use workarounds (sed, scripting etc) You could also use hooks to block running of scripts for some number of turns after an attempt to cheat.
The agent can disable the lints inline, so that's not sufficient.
Also, I haven't found a cross-platform + cross-agent mechanism to set permissions. Much less one that works.
Right now, I'm working on a hook that checks for changes in source files, but the plug-in system (at least of opencode) seems quite buggy.