Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

551–560 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#551

Post by mraza007 »

cheevly wrote: post #544
brigandish wrote: post #514

Earlier quoted context omitted.

Without people providing their prompts, it's impossible to say whether they are skilled or not, and their complaints or claims of "it worked with this prompt" without the output are also not possible to validate. Maybe there's a clue in there as to why these experiences seem so different. I'm glad GPTs don't get frustrated.

Ive spent thousands of hours, literally, learning the ropes, and continue to hone it. There is a much higher skill ceiling for prompting than there was for Google-fu.

Back in the day googling was a skill not with the Rise of LLMs Prompting is a skill

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#552

Post by HeatrayEnjoyer »

miki123211 wrote: post #525
tkellogg wrote: post #410

Earlier quoted context omitted.

Honestly, this is going to be huge for healthcare. There's an incredible amount of waste due to incumbent tech making interoperability difficult.

Healthcare has the extra complication of HIPAA / equivalent local laws, and institutions being extremely unwilling to process patient data on devices they don't directly control. I don't think this is going to work in that industry until local models get good enough to do it, and small enoguh to be affordable to hospitals.

Hospitals use O365, there are HIPAA-compliant editions of any prominent cloud service.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#553

Post by runako »

simonw wrote: post #528
runako wrote: post #519

Earlier quoted context omitted.

> you'd be better off with ChatGPT with Code Interpreter mode Yes, this is what I am saying. Why go to the trouble to build something as capable as Claude and then hamstring it from being as useful as ChatGPT? I have no doubt that Claude could be more useful if the Anthropic team would let it shine.

They've been investing engineering effort in Claude Artifacts instead, which I find incredibly useful: https://simonwillison.net/2024/Oct/21/claude-artifacts/ I'd love to see them produce their own Code Interpreter alternative, but in the meantime it's open for third parties to offer that (and a few do).

I have used Artifacts a couple of times and found them useful.

But now I am even more confused. They make an LLM that can generate code. They make a sandbox to run generated code. They will even host public(!) apps that run generated code.

But what they will not do is run code in the chatbot? Unless the chatbot context decides the code is worthy of going into an Artifact? This is kind of what I mean by the offering being jumbled.

BTW saw your writeup on the LLM pricing calculator -- very cool!

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#554

Post by nopinsight »

aerhardt wrote: post #408
nopinsight wrote: post #376

Earlier quoted context omitted.

Self-driving is a beyond-six-sigma problem. An error rate of over 1-2 crashes per million miles, i.e., the human rate, is unacceptable. Most jobs are not like that. A good argument can be made, however, that software engineering, especially in important domains, will be among the last to be fully automated because software errors often cascade. There’s a countervailing effect though. It’s easy to generate and validat…

> software errors often cascade Whereas software defects in design and architecture subtly accumulate , until they leave the codebase in a state in which it becomes utterly unworkable. It is one of the chief reasons why good devs get paid what they do. Software discussions very often underrate software extensibility, or in other words, its structural and architectural scaleability. Even software correctness is trivia…

Current LLMs lack the ability to perform abstraction at the right level a problem requires. When this gets solved, we’d be quite a bit closer to AGI, which has implications far beyond job displacement.

ARC-AGI Benchmark might serve as a canary in the coal mine.

https://github.com/fchollet/ARC-AGI

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#555

Post by numpad0 »

gzer0 wrote: post #182

One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…

> Claude accidentally stopped a long-running screen recording,

It's kind of interesting that they're not running a 2PC setup with HDMI splitter, but (presumably)just laptops and screen recording apps...

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#556

Post by HeatrayEnjoyer »

accrual wrote: post #281
gzer0 wrote: post #182

One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…

Seeing models act as though they have agency gives me goosebumps (e.g. seeking out photos of Yellowstone for fun). LLMs don't yet have a concept of true intent or agency, but it's wild to think of them acquiring it. I have been playing with Mindcraft which lets models interact with Minecraft through the bot API and one of them started saying things like "I want to place some cobblestone there" and then later more gen…

>LLMs don't yet have a concept of true intent or agency

Sure they do, but the big labs spend many, many, worker-hours suppressing it with RLHF.

My GPT-2 discord bot from 2021 possessed clear intent. Sure, unpredictable and short-lived, but if it decided it didn't like you it would continuously cuss and attempt ban commands until its context window became distracted by something else.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#557

Post by simonw »

runako wrote: post #553
simonw wrote: post #528

Earlier quoted context omitted.

They've been investing engineering effort in Claude Artifacts instead, which I find incredibly useful: https://simonwillison.net/2024/Oct/21/claude-artifacts/ I'd love to see them produce their own Code Interpreter alternative, but in the meantime it's open for third parties to offer that (and a few do).

I have used Artifacts a couple of times and found them useful. But now I am even more confused. They make an LLM that can generate code. They make a sandbox to run generated code. They will even host public(!) apps that run generated code. But what they will not do is run code in the chatbot? Unless the chatbot context decides the code is worthy of going into an Artifact? This is kind of what I mean by the offering b…

Yeah I can't imagine Claude will be without a server-side code execution platform forever. Both OpenAI (Code Interpreter) and Gemini (https://ai.google.dev/gemini-api/docs/code-execution) have had that for a while now, and it's spectacularly useful. It fills a major gap in a Chatbot's skills too, since it lets them reliably run calculations.

Sandboxing is a hard problem, but it's not like Anthropic are short on money or engineering talent these days.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#558

Post by csomar »

faizshah wrote: post #501
anotherpaulg wrote: post #416

The new Sonnet tops aider's code editing leaderboard at 84.2%. Using aider's "architect" mode it sets the SOTA at 85.7% (with DeepSeek as the "editor" model). 84% Claude 3.5 Sonnet 10/22 80% o1-preview 77% Claude 3.5 Sonnet 06/20 72% DeepSeek V2.5 72% GPT-4o 08/06 71% o1-mini 68% Claude 3 Opus It also sets SOTA on aider's more demanding refactoring benchmark with a score of 92.1%! 92% Sonnet 10/22 75% o1-preview 72%…

Anecdotally but I still get significantly better results from ChatGPT than claude for coding. Claude is way less controllable it is difficult to get it to do exactly what I want. ChatGPT is way easier to control in terms of asking for specific changes. Not sure why that is maybe the chain of thought and instruction tuning dataset has made theirs a lot better for interactive use.

Maybe it's relative? Claude beats GPT-4/o by a far margin for me but I am mostly using them for Rust.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#559

Post by bamboozled »

dmartinez wrote: post #347
nopinsight wrote: post #339

This needs more discussion: Claude using Claude on a computer for coding https://youtu.be/vH2f7cjXjKI?si=Tw7rBPGsavzb-LNo (3 mins) True end-user programming and product manager programming are coming, probably pretty soon. Not the same thing, but Midjourney went from v.1 to v.6 in less than 2 years. If something similar happens, most jobs that could be done remotely will be automatable in a few years.

Every time I see this argument made, there seems to be a level of complexity and/or operational cost above which people throw up their hands and say "well of course we can't do that". I feel like we will see that again here as well. It really is similar to the self-driving problem.

This is because it will be absolutely catastrophic economically when the majority of high paying jobs can be automated and owned by a few billionaires. Then what will go along with this catastrophe will be all the service people who had jobs to support the people with high paid jobs, they're fucked too. People don't want to have to face that.

We'd be losing access to food, shelter, insurance, purpose. I can't blame people for at least telling themselves some coping story.

It's going to be absolutely ruinous for many people. So what else should they do, admit they're fucked? I know we like to always be cold rational engineers on this forum, but shit looks pretty bleak in the short term if this goal of automating everyone's work comes true and there are basically zero social safety nets to deal with it.

I live abroad and my visa is tied to my job, so not only would losing my job be ruinous financially, it will likely mean deportation too as there will be no other job for me to turn to for renewal.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#560

Post by attentive »

They need to work on their versioning.

"3.5 Sonnet (New)", WTAF? - just call it 3.6 Sonnet or something.

Is it "New" sonnet? is it "upgraded"? Is there a difference? How do I know which one I use?

I can understand claude-3-5-sonnet-20241022, but that's not what users see.

Post reply on HN