Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on the new Claude analysis JavaScript code execution tool

simonwillison.net

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Notes on the new Claude analysis JavaScript code execution tool

#41

Post by mattmanser »

dartos wrote: post #38
TheRealPomax wrote: post #14

Earlier quoted context omitted.

Isn't what how all JS runs in the browser? There are different restrictions based on where JS comes from, and what context it gets loaded into.

All browser js runs in a browser sandbox and, by default, none of it needs to be explicitly trusted in most browsers. I don’t think there are very many restrictions on what js can do on a given page. At least none come to mind. Not really sure you mean by “context” either. Maybe service workers? Unless you’re talking about loading js within iframes… but that’s a different can of worms.

You've misunderstood the GP's question. If you read the other answers you might understand what he's asking. Hence exactly why they're all talking about iframes.

You used to be able to do it quite easily, but it meant people could essentially impersonate the user if you got them to execute some javascript. So having a code editor would be a recipe for account hijacking.

So gradually browsers locked it all down. Long gone are the days of just doing 'eval()'. In the 2000s I worked on code where we actually did that!

Ah, the days of getting away with massive security holes that no-one even knew how to exploit.

Re: Notes on the new Claude analysis JavaScript code execution tool

#42

Post by dartos »

mattmanser wrote: post #41
dartos wrote: post #38

Earlier quoted context omitted.

All browser js runs in a browser sandbox and, by default, none of it needs to be explicitly trusted in most browsers. I don’t think there are very many restrictions on what js can do on a given page. At least none come to mind. Not really sure you mean by “context” either. Maybe service workers? Unless you’re talking about loading js within iframes… but that’s a different can of worms.

You've misunderstood the GP's question. If you read the other answers you might understand what he's asking. Hence exactly why they're all talking about iframes. You used to be able to do it quite easily, but it meant people could essentially impersonate the user if you got them to execute some javascript. So having a code editor would be a recipe for account hijacking. So gradually browsers locked it all down. Long…

> If you read the other answers you might understand what he's asking

Dude, relax. There were no other comments when I asked…

Re: Notes on the new Claude analysis JavaScript code execution tool

#44

Post by mannanj »

simonw wrote: post #3

I've been trying to figure out the right pattern for running untrusted JavaScript code in a browser sandbox that's controlled by a page for a while now, looks like Anthropic have figured that out. Hoping someone can reverse engineer exactly how they are doing this - their JavaScript code is too obfuscated for me to dig out the tricks, sadly.

commenting to save this for later

Re: Notes on the new Claude analysis JavaScript code execution tool

#45

Post by emmanueloga_ »

bravura wrote: post #40
thenaturalist wrote: post #8

Funnily enough, I test code generation both on unpaid Claude and ChatGPT. When working with Python, I've found Sonnet (pre 3.5) to be quite superior to ChatGPT (mostly 4, sometimes 3.5) with regards to verbosity, structure and prompt / instruct comprehension. I've switched to a JavaScript project two weeks ago and the tables have turned. Sonnet 3.5 is much more verbose and I need to make corrections a few times, wher…

Don't call me crazy (I am actually), but sometimes I will keep both ChatGPT and Claude open side-by-side and use them to audit each other. I'll give them the same prompt. When they respond, re-prompt with: "What are your thoughts on this approach? Pros and cons. Integrate the best ideas from both: [answer from the other model]" Repeat until total satisfaction or frustration is achieved.

This is similar to what Aider does in "architect" mode [1].

--

1: https://aider.chat/docs/usage/modes.html#architect-mode-and-...

Re: Notes on the new Claude analysis JavaScript code execution tool

#47

Post by dakotasmith »

andai wrote: post #23
advaith08 wrote: post #22

The custom instructions to the model say: "Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity." They seem to be apologizing to the model in the system prompt?? This is so intriguing

Has anyone looked into the effect of politeness on performance?

Large Language Models Understand and Can Be Enhanced by Emotional Stimuli

https://arxiv.org/abs/2307.11760

Re: Notes on the new Claude analysis JavaScript code execution tool

#48

Post by trillic »

cruffle_duffle wrote: post #37
Me1000 wrote: post #29

Earlier quoted context omitted.

This is the real value here. Keeping a secure environment to run untrusted code along side user data is a real liability for them. It's not their core competency either, so they can just lean on browser sandboxing and not worry about it.

How is doing it server side a different challenge than something like google collab or any of those Jupyter notebook type services?

Google Collab are all individual VMs. It seems Anthropic doesn’t want to be in the “host a VM for every single user” business.

Re: Notes on the new Claude analysis JavaScript code execution tool

#49

Post by rajnathani »

qeternity wrote: post #20
animal_spirits wrote: post #2

That's an interesting idea to generate javascript and execute it client side rather than server side. I'm sure that saves a ton of money for Anthropic not by not having to spin up a server for each execution.

The cost savings for this are going to be a rounding error. I imagine this is a broader push to be able to have Claude pilot your browser (and other applications) in the future. This is the right way to go about it versus having a headless agent: users can be in the loop and you can bootstrap and existing environment. Otoh it’s going to be a security nightmare.

The cost-savings would actually be significant. Spinning up a sandboxed container/VM or chroot jail a thousand times a month for a user paying a $20/month, when you already as a company have huge GPU bills on the training and inference side and NRE costs, would be gaping.

Re: Notes on the new Claude analysis JavaScript code execution tool

#50

Post by singularity2001 »

mannanj wrote: post #44
simonw wrote: post #3

I've been trying to figure out the right pattern for running untrusted JavaScript code in a browser sandbox that's controlled by a page for a while now, looks like Anthropic have figured that out. Hoping someone can reverse engineer exactly how they are doing this - their JavaScript code is too obfuscated for me to dig out the tricks, sadly.

commenting to save this for later

I used this technique until someone told me that you can use the upvote arrow and find these in your profile
Post reply on HN