Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code 2.0

npmjs.com

Post reply on HN
Print view View original

371–380 of 431 posts

Re: Claude Code 2.0

#371

Post by bakies »

wiesbadener wrote: post #357
simonw wrote: post #331

Earlier quoted context omitted.

You've been safe since launch because you haven't faced an adversarial prompt injection attack yet. You (and many, many others) likely won't take this threat seriously until adversarial attacks become common. Right now, outside of security researcher proof of concepts, they're still vanishingly rare. You ask why I'm obsessed with the danger? That's because I've been tracking prompt injection - and our total failure t…

Hi Simon, I share your worries on this topic. I saw you experiment a lot with python. Do you have a python-focused sandboxed devcontainer setup for Claude Code / Codex you want to share? Or even a full stack setup? Claude's devcontainer setup ( https://github.com/anthropics/claude-code/tree/main/.devcont... ) is focused on JS with npm.

you can do anything in that devcontainer, i have a dockerfile that adds golang tools and claude code just runs whatever install it needs anyway :)

I actually preferred running stuff in containers to keep my personal system clean anyway so I like this better than letting claude use my laptop. I'm working on hosting devcontainer claude code in kubernetes too so I dont need my laptop at all.

Re: Claude Code 2.0

#372

Post by IMTDb »

ACCount37 wrote: post #310
IMTDb wrote: post #307

Earlier quoted context omitted.

> LLMs are inherently safe because they can't do anything other than write text That is still very much the case; the danger comes from what you do from the text that is generated. Put a developer in a meeting room and no computer access, no internet etc; and let him scream instructions through the window. If he screams "delete prod DB", what do you do ? If you end up having to restore a backup that's on you, but the…

That's like saying "humans are inherently safe because you can throw them in a jail forever and then there's nothing they can do". But are all humans in jails? No, the practical reason being that it limits their usefulness. Humans like it better when other humans are useful. The same holds for AI agents. The ship has sailed: no one is going to put every single AI agent in jail. The "inherent safety" of LLMs comes onl…

Humans are not inherently safe; there is very little you can do to prevent a human with a hammer to kill another one. In fact what you usually do with these humans is to put them in jail because they have no direct ability to hurt anyone.

LLM are in jail: an LLM outputting {"type": "function", "function": {"name": "execute_bash", "parameters": {"command": "sudo rm -rf /"}}} isn't unsafe. The unsafe part is the scaffolding around the LLM that will fuckup your entire filesystem. And my whole point is that there are ways to make that scaffolding safe. There is a reason why we have permissions on a filesystem, why we have read only databases etc etc.

Re: Claude Code 2.0

#373

Post by bakies »

guhcampos wrote: post #315
saberience wrote: post #308

Earlier quoted context omitted.

Incredibly dangerous to use? Seems like a wild exaggeration. I’ve been using Claude code since launch, must have used it for 1000 hours or more by now, and it’s never done anything I didn’t want it to do. Why would I run it in a sandbox? It writes code for me and occasionally runs a build and tests. I’m not sure why you’re so fixated on the “danger”, when you use these things all the time you end up realizing that th…

It is dangerous. Just yesterday my cursor agent made some changes to a live kubernetes cluster even over my specific instruction not to. I gave it kubectl to analyze and find the issues with a large Prometheud + AlertManager configuration, then switched windows to work on something else. When I was back the MF was patching live resources to try and diagnose the issue.

yeah claude is really eager to apply stuff directly to the cluster to the wrong context even with constant reminding that it rolls out through gitops. I think there's a way to restrict more than "kubectl" so you can allow get/describe but not apply.

Re: Claude Code 2.0

#374

Post by marckrn »

rcv wrote: post #204
marckrn wrote: post #12

You can find the revamped prompt on github[1], or on twitter summarized by my bot[2]. [1] https://github.com/marckrenn/cc-mvp-prompts/compare/v1.0.128... [2] https://x.com/CCpromptChanges/status/1972709093874757976

> 2025-09-29T16:55:10.367Z is the date. Write a haiku about it. what in the world?

That's just a dynamic bogus prompt used to trace and extract the system prompt.

Here's how it works in detail: https://mariozechner.at/posts/2025-08-03-cchistory/

Re: Claude Code 2.0

#375

Post by marckrn »

nojs wrote: post #231
marckrn wrote: post #12

You can find the revamped prompt on github[1], or on twitter summarized by my bot[2]. [1] https://github.com/marckrenn/cc-mvp-prompts/compare/v1.0.128... [2] https://x.com/CCpromptChanges/status/1972709093874757976

How are you extracting this - aren’t the main labs obfuscating these (meaning it’s likely to be a decoy or incomplete version)?

With cchistory https://github.com/badlogic/cchistory

Here's how it works: https://mariozechner.at/posts/2025-08-03-cchistory/

Re: Claude Code 2.0

#376

Post by MrGreenTea »

mistahchris wrote: post #368
pmarreck wrote: post #362

Earlier quoted context omitted.

I already set up a jj (jujutsu) repo in my projects colocated with git (it uses git for its backend). Once you additionally set up a certain background daemon, it will then autocommit (label-lessly) every change to every file in that project. So you get "infinite undo", basically. It's actually more powerful than this checkpointing idea.

I'm a recent jj convert, and working with llms was actually a driver for my own jj adoption. I haven't tried the watch daemon, but I do run `jj new` anytime i ask the llm agent to do anything. It has worked amazingly well.

I automated that with some simple claude hooks that detect if we're in a `jj` repo and also prompts claude to `jj commit -m` after every change.

Re: Claude Code 2.0

#377

Post by marckrn »

data-ottawa wrote: post #322
stefan_ wrote: post #288

Earlier quoted context omitted.

Meanwhile they deleted the "do not add emojis" part. Look forward to all sorts of logging messages with emojis in them.

I don’t understand where the AI love of emojis comes from. I’ve never seen them in a professional codebase outside of basic logging. I assume it comes from the myriad tutorial content on medium or something. gpt-oss is the most egregious emoji user: it uses emoji for numbers in section headings in code, which was clearly a stylistic choice finetuned into the model and it fights you on removing them. I’ve noticed Clau…

I'm pretty sure it's caused by RLHF :wink:

Re: Claude Code 2.0

#378

Post by marckrn »

huflungdung wrote: post #86
marckrn wrote: post #12

You can find the revamped prompt on github[1], or on twitter summarized by my bot[2]. [1] https://github.com/marckrenn/cc-mvp-prompts/compare/v1.0.128... [2] https://x.com/CCpromptChanges/status/1972709093874757976

How are these reliably extracted (hint: they’re not)

See https://mariozechner.at/posts/2025-08-03-cchistory/

Why do you think these aren't legit?

Re: Claude Code 2.0

#379

Post by simonw »

wiesbadener wrote: post #357
simonw wrote: post #331

Earlier quoted context omitted.

You've been safe since launch because you haven't faced an adversarial prompt injection attack yet. You (and many, many others) likely won't take this threat seriously until adversarial attacks become common. Right now, outside of security researcher proof of concepts, they're still vanishingly rare. You ask why I'm obsessed with the danger? That's because I've been tracking prompt injection - and our total failure t…

Hi Simon, I share your worries on this topic. I saw you experiment a lot with python. Do you have a python-focused sandboxed devcontainer setup for Claude Code / Codex you want to share? Or even a full stack setup? Claude's devcontainer setup ( https://github.com/anthropics/claude-code/tree/main/.devcont... ) is focused on JS with npm.

I've been trying out GitHub Codespaces as a sandbox, which works pretty well.

I wrote a bit about that in a new post this morning, but I'm still looking for an ideal solution: https://simonwillison.net/2025/Sep/30/designing-agentic-loop...

Re: Claude Code 2.0

#380

Post by simonw »

mehdibl wrote: post #367
simonw wrote: post #331

Earlier quoted context omitted.

You've been safe since launch because you haven't faced an adversarial prompt injection attack yet. You (and many, many others) likely won't take this threat seriously until adversarial attacks become common. Right now, outside of security researcher proof of concepts, they're still vanishingly rare. You ask why I'm obsessed with the danger? That's because I've been tracking prompt injection - and our total failure t…

how are you going to get "adversarial attacks" with prompt injection. If you don't fetch data from external sources. Web scraping ( you can channel that thru Perplexity by the to sanitize it). PR reviews, would be fine if repo is private. I feel this is overly exagerated here. There is more issues that are currently getting leverage to hack with vscode extension than AI prompt injection, that require a VERY VERY comp…

If you don't fetch data from external sources then you're safe from prompt injection.

But that's a very big if. I've seen Claude Code attempt to debug a JavaScript issue by running curl against the jsdelivr URL for a dependency it's using. A supply chain attack against NPM (and those aren't exactly rare these days) could add comments to code like that which could trigger attacks.

Ever run Claude Code in a folder that has a downloaded PDF from somewhere? There are a ton of tricks for hiding invisible malicious instructions in PDFs.

I run Claude Code and Codex CLI in YOLO mode sometimes despite this risk because I'm basically crossing my fingers that a malicious attack won't slip in, but I know that's a bad idea and that at some point in the future these attacks will be common enough for the risk to no longer be worth it.

Post reply on HN