Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

461–470 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#461

Post by trzy »

mrmansano wrote: post #399
trzy wrote: post #279

Pretty cool! I use Claude 3.5 to control a robot (ARKit/iOS based) and it does surprisingly well in the real world: https://youtu.be/-iW3Vzzr3oU?si=yzu2SawugXMGKlW9

That looks pretty cool, congrats! How feasible is it to be a product by itself? Did you try with a local edge model?

None of the small LLMs are good enough yet. You could certainly build a system around local VLMs but it would require much more task specific programming baked in.

I’m certainly interested in building a product (not entirely controlled by an LLM but I see lots of utility in building interfaces with them) but not really sure what this would be useful for. Looking into some spaces now but there has to be a clear ROI to get any sort of funding for robotics.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#462

Post by botro »

nomel wrote: post #273
Hizonner wrote: post #37

Earlier quoted context omitted.

It's just bizarre to force a computer to go through a GUI to use another computer. Of course it's going to be expensive.

Not at all! Programs, and websites, are built for humans , and very very rarely offer non-GUI access. This is the only feasible way to make something useful now. I think it's also the reason why robots will look like humans, be the same proportions as humans, have roughly the same feet and hands as humans: everything in the world was designed for humans. That being the foundation is going to influence what's built on…

And to take a historic analogy, cars today are as wide as they are because that's about how wide a single lane roadway is. And a single lane roadway is as wide as it is because that's about the width of two horses drawing a carriage.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#463

Post by TaylorAlexander »

bongodongobob wrote: post #298
TaylorAlexander wrote: post #216

And today I realized that despite it being an extremely common activity, we don’t really have a word for “using the computer” which is distinct from “computing”. It’s funny because AI models are always “using a computer” but now they can “use your computer.”

Operating a computer?

Right. We don’t have a word for that. Like “using a bicycle” has the word “bicycling”.

Tho someone here suggested “computering” which is pretty good.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#464

Post by mewpmewp2 »

claytongulick wrote: post #459
girvo wrote: post #419

Earlier quoted context omitted.

We’ll see. Having worked in this space in the past, the technical challenges are able to overcome today with no new technology: its a business sales and regulation challenge more than a tech one.

Sometimes. In my case I have a bunch of nurses that waste a huge amount of time dealing with clerical work and tech hoops, rather than operating at the top of their license. Traditional RPAs are tough when you're dealing with VPNs, 2fa, remote desktop (in multiple ways), a variety of EHRs and scraping clinical documentation from poorly structured clinical notes or PDFs. This technology looks like it could be a game c…

True, 2FA and all these little details that exist now have made this automation quite insanely complicated. It is of course necessary that we have 2FA etc, but there is huge potential in solving this I believe.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#465

Post by TacticalCoder »

nopinsight wrote: post #339

This needs more discussion: Claude using Claude on a computer for coding https://youtu.be/vH2f7cjXjKI?si=Tw7rBPGsavzb-LNo (3 mins) True end-user programming and product manager programming are coming, probably pretty soon. Not the same thing, but Midjourney went from v.1 to v.6 in less than 2 years. If something similar happens, most jobs that could be done remotely will be automatable in a few years.

> This needs more discussion:

"Create a simple website" has to be one of the most common blog / example out there in about every programming language.

It can automate stuff? That's cool: I already did automate screenshots and then AI looking if it looks like phishing or not (and it's quite good at it).

I mean: the "Claude using Claude" may seem cool, but I dispute the "for coding" part. That's trivial stuff. A trivial error (which it doesn't fix btw: it just deletes everything).

'Claude, write me code to bring SpaceX rockets back on earth"

or

"Claude, write me code to pilot a machine to treat a tumor with precision"

This was not it.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#466

Post by HarHarVeryFunny »

The "computer use" ability is extremely impressive!

This is a lot more than an agent able to use your computer as a tool (and understanding how to do that) - it's basically an autonomous reasoning agent that you can give a goal to, and it will then use reasoning, as well as it's access to your computer, to achieve that goal.

Take a look at their demo of using this for coding.

https://www.youtube.com/watch?v=vH2f7cjXjKI

This seems to be an OpenAI GPT-o1 killer - it may be using an agent to do reasoning (still not clear exactly what is under the hood) as opposed to GPT-o1 supposedly being a model (but still basically a loop around an LLM), but the reasoning it is able to achieve in pursuit of a real world goal is very impressive. It'd be mind boggling if we hadn't had the last few years to get used to this escalation of capabilities.

It's also interesting to consider this from POV of Anthropic's focus on AI safety. On their web site they have a bunch of advice on how to stay safe by sandboxing, limiting what it has access to, etc, but at the end of the day this is a very capable AI able to use your computer and browser to do whatever it deems necessary to achieve a requested goal. How far are we from paperclip optimization, or at least autonomous AI hacking ?

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#467

Post by SoftTalker »

nopinsight wrote: post #376
dmartinez wrote: post #347

Earlier quoted context omitted.

Every time I see this argument made, there seems to be a level of complexity and/or operational cost above which people throw up their hands and say "well of course we can't do that". I feel like we will see that again here as well. It really is similar to the self-driving problem.

Self-driving is a beyond-six-sigma problem. An error rate of over 1-2 crashes per million miles, i.e., the human rate, is unacceptable. Most jobs are not like that. A good argument can be made, however, that software engineering, especially in important domains, will be among the last to be fully automated because software errors often cascade. There’s a countervailing effect though. It’s easy to generate and validat…

Errors not only cascade, in certain cases they have global impact in very little time. E.g. CrowdStrike.

And what is the title element on CrowdStrike's website today? "CrowdStrike: We Stop Breaches with AI-native Cybersecurity"

Can't wait.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#468

Post by vl »

aerhardt wrote: post #408
nopinsight wrote: post #376

Earlier quoted context omitted.

Self-driving is a beyond-six-sigma problem. An error rate of over 1-2 crashes per million miles, i.e., the human rate, is unacceptable. Most jobs are not like that. A good argument can be made, however, that software engineering, especially in important domains, will be among the last to be fully automated because software errors often cascade. There’s a countervailing effect though. It’s easy to generate and validat…

> software errors often cascade Whereas software defects in design and architecture subtly accumulate , until they leave the codebase in a state in which it becomes utterly unworkable. It is one of the chief reasons why good devs get paid what they do. Software discussions very often underrate software extensibility, or in other words, its structural and architectural scaleability. Even software correctness is trivia…

Fundamentally there is human with limited brain capacity that got trained to that. It’s just a question of time when there are equally capable, and then exceedingly capable models. There is nothing magical or special about human brain.

The only question is how fast it is going to happen. Ie what percentage of jobs is going to be replaced next year and so on.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#469

Post by joshuamcginnis »

Is there anything out there yet that will let me issue the command:

> Refactor the api folder with any recommended readability improvements or improvements that would help DRY up code without adding additional complexity.

Then I can just `git status` to see the changes?

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#470

Post by skissane »

astrange wrote: post #391
skissane wrote: post #350

Earlier quoted context omitted.

You don’t know for a fact that those two specific packages don’t have supported APIs. Just because the user doesn’t know of any API doesn’t mean none exists. The average accountant or doctor is never going to even ask the vendor “is there an API” because they wouldn’t know what to do with one if there was.

If they're accessible to screen readers they have one. Accessibility is API for apps in disguise. In this case I doubt they're networked apps so they probably don't have a server API.

> In this case I doubt they're networked apps so they probably don't have a server API.

I think it would be very unusual this decade for software used to run either a medical practice or tax accountants to not be networked. Most such practices have multiple doctors/accountants, each with their individual computer, and they want to be able to share files, so that if your doctor/accountant is away their colleague can attend to you. Managing backups/security/etc is all a lot easier when the data is stored in a central server (whether in the cloud or a closet) than on individual client machines.

Just because it is a fat client MFC-based Windows app doesn’t mean the data has to be stored locally. DCOM has been a thing since 1996.

Post reply on HN