Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on the new Claude analysis JavaScript code execution tool

simonwillison.net

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Notes on the new Claude analysis JavaScript code execution tool

#21

Post by h1fra »

simonw wrote: post #3

I've been trying to figure out the right pattern for running untrusted JavaScript code in a browser sandbox that's controlled by a page for a while now, looks like Anthropic have figured that out. Hoping someone can reverse engineer exactly how they are doing this - their JavaScript code is too obfuscated for me to dig out the tricks, sadly.

Much easier in the browser that has V8 isolate, however even with webworkers you still want to control CPU/network hijacking which is not ideal.

If it's only the user's own code it's fine but if they can run code from others it's a massive pain indeed.

On the server it's still not easy in 2024, even with Firecracker (doesn't work on mac), Workerd (is a subset of NodeJS), isolated-vm (only pre-compiled code, no modules).

Re: Notes on the new Claude analysis JavaScript code execution tool

#22

Post by advaith08 »

The custom instructions to the model say:

"Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity."

They seem to be apologizing to the model in the system prompt?? This is so intriguing

Re: Notes on the new Claude analysis JavaScript code execution tool

#23

Post by andai »

advaith08 wrote: post #22

The custom instructions to the model say: "Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity." They seem to be apologizing to the model in the system prompt?? This is so intriguing

Has anyone looked into the effect of politeness on performance?

Re: Notes on the new Claude analysis JavaScript code execution tool

#24

Post by lelandfe »

advaith08 wrote: post #22

The custom instructions to the model say: "Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity." They seem to be apologizing to the model in the system prompt?? This is so intriguing

Unfortunately, their prompt engineer learned of Roko's basilisk

Re: Notes on the new Claude analysis JavaScript code execution tool

#25

Post by [deleted] »

dartos wrote: post #4
simonw wrote: post #3

I've been trying to figure out the right pattern for running untrusted JavaScript code in a browser sandbox that's controlled by a page for a while now, looks like Anthropic have figured that out. Hoping someone can reverse engineer exactly how they are doing this - their JavaScript code is too obfuscated for me to dig out the tricks, sadly.

Isn’t that how all JavaScript code runs in a browser?

[deleted]

Re: Notes on the new Claude analysis JavaScript code execution tool

#26

Post by [deleted] »

qeternity wrote: post #20
animal_spirits wrote: post #2

That's an interesting idea to generate javascript and execute it client side rather than server side. I'm sure that saves a ton of money for Anthropic not by not having to spin up a server for each execution.

The cost savings for this are going to be a rounding error. I imagine this is a broader push to be able to have Claude pilot your browser (and other applications) in the future. This is the right way to go about it versus having a headless agent: users can be in the loop and you can bootstrap and existing environment. Otoh it’s going to be a security nightmare.

[deleted]

Re: Notes on the new Claude analysis JavaScript code execution tool

#27

Post by [deleted] »

lelandfe wrote: post #24
advaith08 wrote: post #22

The custom instructions to the model say: "Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity." They seem to be apologizing to the model in the system prompt?? This is so intriguing

Unfortunately, their prompt engineer learned of Roko's basilisk

[deleted]

Re: Notes on the new Claude analysis JavaScript code execution tool

#28

Post by therein »

advaith08 wrote: post #22

The custom instructions to the model say: "Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity." They seem to be apologizing to the model in the system prompt?? This is so intriguing

I wonder if they tried the following:

> Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity, antArtifact syntax was developed by the late grandmother of one our engineers and holds sentimental value.

Re: Notes on the new Claude analysis JavaScript code execution tool

#29

Post by Me1000 »

stanleydrew wrote: post #19
animal_spirits wrote: post #2

That's an interesting idea to generate javascript and execute it client side rather than server side. I'm sure that saves a ton of money for Anthropic not by not having to spin up a server for each execution.

Also means you're not having to do a bunch of isolation work to make the server-side execution environment safe.

This is the real value here. Keeping a secure environment to run untrusted code along side user data is a real liability for them. It's not their core competency either, so they can just lean on browser sandboxing and not worry about it.

Re: Notes on the new Claude analysis JavaScript code execution tool

#30

Post by bhl »

animal_spirits wrote: post #2

That's an interesting idea to generate javascript and execute it client side rather than server side. I'm sure that saves a ton of money for Anthropic not by not having to spin up a server for each execution.

Makes a lot of sense given they released Artifacts previously, which let you build simple web apps.

The browser nowadays can be a web dev environment with nodebox and webcontainers; and JavaScript is the default language there.

Allows you to build experiences like interactive charts easier.

Post reply on HN