Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code 2.0

npmjs.com

Post reply on HN
Print view View original

301–310 of 431 posts

Re: Claude Code 2.0

#301

Post by nextaccountic »

budududuroiu wrote: post #289
simonw wrote: post #49

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this - maybe just "coding agents" instead) is that they're actually an interesting form of general agent . Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They're incredibly dangerous to use if you d…

I’m experimenting with Nix shells for this tool isolation and whitelisting

That's not enough for security. Morally it should be - there's no reason we shouldn't be able to run untrusted software easily - but it won't have a firewall for example

Maybe something like bubblewrap could help

Re: Claude Code 2.0

#302

Post by NiloCK »

nojs wrote: post #276
libraryofbabel wrote: post #162

Earlier quoted context omitted.

No, that's not the point of this new checkpoints feature. It's already been possible for a while to rewind context in Claude Code by pressing . This feature rewinds code state alongside context : > Our new checkpoint system automatically saves your code state before each change , and you can instantly rewind to previous versions by tapping Esc twice or using the /rewind command. https://www.anthropic.com/news/enablin…

From the docs it looks like this feature only reverts the edit tool calls, and not e.g. bash commands that have been executed: > Checkpoints apply to Claude’s edits and not user edits or bash commands, and we recommend using them in combination with version control

How could they possibly hope to undo bash commands, whose side effects could be anything, anywhere?

Hey Claude... uh... unlaunch those

Re: Claude Code 2.0

#303

Post by ACCount37 »

ozgung wrote: post #295
simonw wrote: post #49

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this - maybe just "coding agents" instead) is that they're actually an interesting form of general agent . Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They're incredibly dangerous to use if you d…

> Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. One criticism on current generation of AI is that they have no real world experience. Well, they have enormous amount of digital world experience. That, actually, has more economical value.

They have a lot of secondhand knowledge and very little firsthand knowledge. RLVR works so well because it's a way to give LLMs some of the latter.

Re: Claude Code 2.0

#304

Post by [deleted] »

pmarreck wrote: post #195

I was already using jj (jujutsu) to do my own rewinds (it saves every change to every file as an unlabeled commit, assuming you set up its daemon). Would sort of prefer to continue to do that since it's far more flexible than checkpoints

[deleted]

Re: Claude Code 2.0

#305

Post by lukaslalinsky »

pmarreck wrote: post #195

I was already using jj (jujutsu) to do my own rewinds (it saves every change to every file as an unlabeled commit, assuming you set up its daemon). Would sort of prefer to continue to do that since it's far more flexible than checkpoints

How do you use jj to get those checkpoints? I was experimenting with jj and claude code, but it was frustrating to have it run jj status all the time, could as well tell it to do git commit all the time.

Re: Claude Code 2.0

#306

Post by swaits »

pmarreck wrote: post #195

I was already using jj (jujutsu) to do my own rewinds (it saves every change to every file as an unlabeled commit, assuming you set up its daemon). Would sort of prefer to continue to do that since it's far more flexible than checkpoints

Checkpoints also include context.

I also use jj to checkpoint. When working on a change, each time I get to a stable point I squash and start fresh with an empty change.

You can absolutely continue doing that.

Re: Claude Code 2.0

#307

Post by IMTDb »

ACCount37 wrote: post #300
simonw wrote: post #49

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this - maybe just "coding agents" instead) is that they're actually an interesting form of general agent . Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They're incredibly dangerous to use if you d…

Back in 2022, when ChatGPT was new, quite a few people were saying "LLMs are inherently safe because they can't do anything other than write text". Some must have even believed what they were saying. Clearly not. Just put an LLM into some basic scaffolding and you get an agent. And as capabilities of those AI agents grow, so would the degree of autonomy people tend to give them.

> LLMs are inherently safe because they can't do anything other than write text

That is still very much the case; the danger comes from what you do from the text that is generated.

Put a developer in a meeting room and no computer access, no internet etc; and let him scream instructions through the window. If he screams "delete prod DB", what do you do ? If you end up having to restore a backup that's on you, but the dude inherently didn't do anything remotely dangerous.

The problem is that the scaffolding people put around LLM is very weak, the equivalent of saying "just do to everything the dude is telling, no question asked, no double check in between, no logging, no backups". There's a reason our industry has development policies, 4 eyes principles, ISO/SOC standards. There already are ways to massively improve the safety of code agents; just put Claude code in a BSD jail and you already have a much safer environment than what 99% of people are doing, this is not that tedious to make. Other safer execution environments (command whitelisting, arguments judging, ...) will be developed soon enough.

Re: Claude Code 2.0

#308

Post by saberience »

simonw wrote: post #49

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this - maybe just "coding agents" instead) is that they're actually an interesting form of general agent . Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They're incredibly dangerous to use if you d…

Incredibly dangerous to use? Seems like a wild exaggeration.

I’ve been using Claude code since launch, must have used it for 1000 hours or more by now, and it’s never done anything I didn’t want it to do.

Why would I run it in a sandbox? It writes code for me and occasionally runs a build and tests.

I’m not sure why you’re so fixated on the “danger”, when you use these things all the time you end up realizing that the safety aspect is really nowhere near as bad as the “AI doomers” seem to make out.

Re: Claude Code 2.0

#309

Post by wrasee »

numpad0 wrote: post #284

fyi: for chatboxes that may take CJK inputs, you MUST use "shift+enter to send" pattern. There is a reason why most multinational chat/LLM app providers always do that instead of simple enter to send even for single-line chatboxes; because plain enter to send breaks input for CJK users. Specifically, Input Method Editors needed for CJK inputs(esp. for C and J), to convert ambiguous semi-readable forms into proper rea…

What’s CJK input? I’m guessing Chinese Japanese Korean?

Re: Claude Code 2.0

#310

Post by ACCount37 »

IMTDb wrote: post #307
ACCount37 wrote: post #300

Earlier quoted context omitted.

Back in 2022, when ChatGPT was new, quite a few people were saying "LLMs are inherently safe because they can't do anything other than write text". Some must have even believed what they were saying. Clearly not. Just put an LLM into some basic scaffolding and you get an agent. And as capabilities of those AI agents grow, so would the degree of autonomy people tend to give them.

> LLMs are inherently safe because they can't do anything other than write text That is still very much the case; the danger comes from what you do from the text that is generated. Put a developer in a meeting room and no computer access, no internet etc; and let him scream instructions through the window. If he screams "delete prod DB", what do you do ? If you end up having to restore a backup that's on you, but the…

That's like saying "humans are inherently safe because you can throw them in a jail forever and then there's nothing they can do".

But are all humans in jails? No, the practical reason being that it limits their usefulness. Humans like it better when other humans are useful.

The same holds for AI agents. The ship has sailed: no one is going to put every single AI agent in jail.

The "inherent safety" of LLMs comes only from their limited capabilities. They aren't good enough yet to fail in truly exciting ways.

Post reply on HN