Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on Anthropic's Computer Use Ability

composio.dev

Post reply on HN
Print view View original

21–30 of 108 posts

Re: Notes on Anthropic's Computer Use Ability

#21

Post by danielbln »

acrooks wrote: post #15

I've built a couple of experiments using it so far and it has been really interesting. On one hand, it has really helped me with prototyping incredibly fast. On the other, it is prohibitively expensive today. Essentially you pay per click, in some cases per keystroke. I tried to get it to find a flight for me. So it opened the browser, navigated to Google Flights, entered the origin, destination etc. etc. By the time…

Haiku 3.5 wol be here soon, and will before long support tool use and vision, so that should help a lot with cost.

Re: Notes on Anthropic's Computer Use Ability

#22

Post by danielbln »

_heimdall wrote: post #18

I'm all for the MVP approach and shipping quickly, though I'm really surprised they went with image recognition and tooling for injecting mouse/keyboard events for automating human tasks. I wonder why leveraging accessibility tools for this wouldn't have been a better option. Browsers and operating systems both have pretty comprehensive tooling for accessibility tools like screen readers, and the whole point of those…

I think the reason is that this is the most general implementation. It doesn't need playwright or have access to the DOM or anything else, if it has a screen and mouse/keyboard, then it will work. That's quite powerful (if slow and pricey, at the moment).

Re: Notes on Anthropic's Computer Use Ability

#24

Post by infecto »

_heimdall wrote: post #18

I'm all for the MVP approach and shipping quickly, though I'm really surprised they went with image recognition and tooling for injecting mouse/keyboard events for automating human tasks. I wonder why leveraging accessibility tools for this wouldn't have been a better option. Browsers and operating systems both have pretty comprehensive tooling for accessibility tools like screen readers, and the whole point of those…

Those sound like stop gaps at best. Its pretty clear the intended goal here. APIs are easy to integrate with but most systems in existence only have a visual interface intended for humans.

The end goal here is clear, being able to interface with anything available in the screen.

Re: Notes on Anthropic's Computer Use Ability

#25

Post by nerdix »

namanyayg wrote: post #8

What are some good use cases for this? Something that a business can be built around

It's main use case is making the average office worker feel the same existential dread that some programmers feel when they see a LLM spit out a bunch of code in mere seconds.

Re: Notes on Anthropic's Computer Use Ability

#26

Post by im3w1l »

sunilkumardash9 wrote: post #12
ko_pivot wrote: post #4

At the end of the day, the fundamental dynamic here is human creativity. We are taking a tool, the LLM, and stretching it to its limit. That’s great, but that doesn’t mean we are close to AGI. It means we are AGI.

This is a step towards a human-machine hybrid world. Putting a human in the loop can do wonders. Sure, it is expensive now, but the subsequent iterations will crush it.

Have you heard of Centaur chess? A human and a machine would team up to find the best chess moves against another similar team. It's not a thing anymore. Computers have advanced so much that humans can't really contribute in any meaningful sense.

Re: Notes on Anthropic's Computer Use Ability

#27

Post by jampekka »

nerdix wrote: post #25
namanyayg wrote: post #8

What are some good use cases for this? Something that a business can be built around

It's main use case is making the average office worker feel the same existential dread that some programmers feel when they see a LLM spit out a bunch of code in mere seconds.

And when having to pick up the pieces when someone actually uses the thing.

Re: Notes on Anthropic's Computer Use Ability

#28

Post by blurbleblurble »

ko_pivot wrote: post #4

At the end of the day, the fundamental dynamic here is human creativity. We are taking a tool, the LLM, and stretching it to its limit. That’s great, but that doesn’t mean we are close to AGI. It means we are AGI.

This is an insightful comment, though it just goes to show how rigid the framing is of "natural vs. artificial" or "human vs. machine". None of this stuff has any vitality outside of _some_ relationship or interface with people.

Re: Notes on Anthropic's Computer Use Ability

#29

Post by nilstycho »

It seems like a cheaper intermediate capability would be to give Claude the ability to SSH to your computer or to a cloud container. That would unlock a lot of possibilities, without incurring the cost of the vision model or the difficulty of cursor manipulation.

Does this already exist? If not, would the benefits be lower than I think, or would the costs be higher than I think?

Re: Notes on Anthropic's Computer Use Ability

#30

Post by bamboozled »

im3w1l wrote: post #26
sunilkumardash9 wrote: post #12

Earlier quoted context omitted.

This is a step towards a human-machine hybrid world. Putting a human in the loop can do wonders. Sure, it is expensive now, but the subsequent iterations will crush it.

Have you heard of Centaur chess? A human and a machine would team up to find the best chess moves against another similar team. It's not a thing anymore. Computers have advanced so much that humans can't really contribute in any meaningful sense.

Is the point of your comment to make people feel depressed ?

Either we're going to use these tools to augment our abilities or basically just become wiped out, at least our jobs will be, and there is no plan to provide support for anyone. Maybe the tech will make the transition to a post employment world so swift we don't even feel any negative economic effects at all, but let's see.

Post reply on HN