Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

561–570 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#561

Post by csomar »

I like its lazy approach

> I apologize, but it seems we're having some issues installing Chromium. As an alternative, since I can see Firefox is already installed on the system, would you like to use Firefox instead? Or would you prefer me to try a different approach to installing Chromium?

It seems unable to figure out how to install chormium. Sees the Firefox icon out there and just want to use it. You know, like a real lazy human being.

It seems to insistent when it needs to, though

> I clicked on Firefox and it should start opening. Let me take another screenshot to see if the window has appeared:

> Firefox seems to be taking a moment to start up. Let me wait a few seconds and take another screenshot:

> Let me try clicking Firefox one more time to ensure it starts:

> Let me try one more time to take a screenshot to see if Firefox has opened:

> Let me try starting Firefox using the bash tool as an alternative method:

> Firefox has successfully opened! As per the instructions, I should ignore the setup wizard and instead click directly on the address bar. Would you like me to do anything specific with Firefox now that it's open?

I didn't instruct him to ignore the setup wizard. So my guess is that Anthropic has configured it to ignore stuff happening on the screen so it doesn't go loose.

And here he goes through my website, through my hacker news account and then find this very comment

> Looking at his first/most recent comment, it's about a discussion of Claude and computer use. Here's what he wrote:

"I like its lazy approach"

This appears to be a humorous response in a thread about "Computer use, a new Claude 3.5 Sonnet, and Claude..." where he's commenting on an AI's behavior in a situation. The comment is very recent (shown as "8 minutes ago" in the screenshot) and is referring to a situation where an AI seems to have taken a simpler or more straightforward approach to solving a problem.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#562

Post by bamboozled »

hawk_ wrote: post #433
aerhardt wrote: post #408

Earlier quoted context omitted.

> software errors often cascade Whereas software defects in design and architecture subtly accumulate , until they leave the codebase in a state in which it becomes utterly unworkable. It is one of the chief reasons why good devs get paid what they do. Software discussions very often underrate software extensibility, or in other words, its structural and architectural scaleability. Even software correctness is trivia…

I hear you. But I have wondered if there won't be a need to maintain certain like of software when you can just have it be rewritten for each iteration. Like some kind of schema evolution, yes but throwaway software at each iteration.

The challenge might be around the edges here, I guess you'd be able to instruct an agent to always code to a certain API spec, but no piece of software runs or does anything really useful in vacuum.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#563

Post by carlheinzc »

csomar wrote: post #561

I like its lazy approach > I apologize, but it seems we're having some issues installing Chromium. As an alternative, since I can see Firefox is already installed on the system, would you like to use Firefox instead? Or would you prefer me to try a different approach to installing Chromium? It seems unable to figure out how to install chormium. Sees the Firefox icon out there and just want to use it. You know, like a…

Now that was most excellent.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#564

Post by kqr »

csomar wrote: post #558
faizshah wrote: post #501

Earlier quoted context omitted.

Anecdotally but I still get significantly better results from ChatGPT than claude for coding. Claude is way less controllable it is difficult to get it to do exactly what I want. ChatGPT is way easier to control in terms of asking for specific changes. Not sure why that is maybe the chain of thought and instruction tuning dataset has made theirs a lot better for interactive use.

Maybe it's relative? Claude beats GPT-4/o by a far margin for me but I am mostly using them for Rust.

I also think there are subtle differences in how models like to be prompted, so some people will have more luck with one type of model.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#565

Post by visarga »

CSMastermind wrote: post #415
aduffy wrote: post #374

Earlier quoted context omitted.

This has existed for a long time, it's called "RPA" or Robotic Process Automation. The biggest incumbent in this space is UiPath, but there are a host of startups and large companies alike that are tackling it. Most of the things that RPA is used for can be easily scripted, e.g. download a form from one website, open up Adobe. There are a lot of startups that are trying to build agentic versions of RPA, I'm glad to s…

RPA has been a huge pain to work with. It's almost always a framework around existing tools like Selenium that you constantly have to fight against to get good results from. I was always left with the feeling that I could build something better myself just handrolling the scripts rather than using their frameworks. Getting Claude integrated into the space is going to be a game changer.

Most RPA work is in dealing with errors and exceptions, not the "happy path". I don't see how Claude's Screen Agent is going to work out there - what do you do when an error pops up and you need to implement specific business logic how to respond? How about consistency over many executions, and enterprise accounts. You want a centralized way to control agent behavior. Scripting based RPA is also much faster and cheaper to run, and more consistent.

Maybe Anthropic should focus on building a flexible RPA primitive we could use to make RPA workflows with, like for example extracting values from components that need scrolling, selecting values from long drop-down menus, or handling error messages under form fields.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#566

Post by claytongulick »

dimitri-vs wrote: post #479
claytongulick wrote: post #458

Earlier quoted context omitted.

It is. CTO of healthcare org here. I just put a hold on a new RPA project to keep an eye on this and see how it develops. According to their docs, Anthropic will sign a BAA.

Out of curiosity, how are high risk liability enviroments like yours coming to terms with the non-deterministic nature of models like these? Eg. the non-zero chance that it might click a button it *really* shouldn't as demonstrated in the failure demo.

I don't know yet. We may not do it.

We haven't deployed a model like this, it's new.

I've done a ton of various RPAs over the years, using all the normal techniques, and they're always brittle and sensitive to minor updates.

For this, I'm taking a "wait and see" approach. I want to see and test how well it performs in the real world before I deploy it, and wait for it to come out of beta so Anthropic will sign a BAA.

The demo is impressive enough that I want to give the tech a chance to mature before my team and I invest a ton of time into a more traditional RPA.

At a minimum, if we do end up using it, we'll have solid guard rails in place - it'll run on an isolated VM, all of its user access will be restricted to "read only" for external systems, and any content that comes from it will go through review by our nurses.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#567

Post by lobochrome »

csomar wrote: post #561

I like its lazy approach > I apologize, but it seems we're having some issues installing Chromium. As an alternative, since I can see Firefox is already installed on the system, would you like to use Firefox instead? Or would you prefer me to try a different approach to installing Chromium? It seems unable to figure out how to install chormium. Sees the Firefox icon out there and just want to use it. You know, like a…

SYSTEM_PROMPT = f""" * You are utilising an Ubuntu virtual machine using {platform.machine()} architecture with internet access. * You can feel free to install Ubuntu applications with your bash tool. Use curl instead of wget. * To open firefox, please just click on the firefox icon. Note, firefox-esr is what is installed on your system. * Using bash tool you can start GUI applications, but you need to set export DISPLAY=:1 and use a subshell. For example "(DISPLAY=:1 xterm &)". GUI apps run with bash tool will appear within your desktop environment, but they may take some time to appear. Take a screenshot to confirm it did. * When using your bash tool with commands that are expected to output very large quantities of text, redirect into a tmp file and use str_replace_editor or `grep -n -B -A ` to confirm output. * When viewing a page it can be helpful to zoom out so that you can see everything on the page. Either that, or make sure you scroll down to see everything before deciding something isn't available. * When using your computer function calls, they take a while to run and send back to you. Where possible/feasible, try to chain multiple of these calls all into one function calls request. * The current date is {datetime.today().strftime('%A, %B %-d, %Y')}.

* When using Firefox, if a startup wizard appears, IGNORE IT. Do not even click "skip this step". Instead, click on the address bar where it says "Search or enter address", and enter the appropriate search term or URL there. * If the item you are looking at is a pdf, if after taking a single screenshot of the pdf it seems that you want to read the entire document instead of trying to continue to read the pdf from your screenshots + navigation, determine the URL, use curl to download the pdf, install and use pdftotext to convert it to a text file, and then read that text file directly with your StrReplaceEditTool. """

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#568

Post by ianeigorndua »

stavros wrote: post #491
ianeigorndua wrote: post #431

Earlier quoted context omitted.

Are these synthetic or real-world benchmarks? Answering myself: ”Aider’s code editing benchmark asks the LLM to edit python source files to complete 133 small coding exercises from Exercism” Not gonna start looking for a job any time soon

I use Claude for coding and it's fantastic. I definitely have outsourced a lot of my coding to it.

Sure it can do coding but can it do software engineering

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#569

Post by quantadev »

luigipederzani wrote: post #359
quantadev wrote: post #199

Earlier quoted context omitted.

I think the best use case for AI `Computer Use` would be a simple positioning of the mouse and asking for conformation before a click. For most use cases this is all people will want/need. If you don't know how to do something, it is basically teaching you how, in this case, rather than taking full control and doing things so fast you don't have time to stop of going rogue.

I totally agree with you. At orango.ai, we have implemented the auto-click feature, but before it clicks, we position the cursor on the button and display a brief loading animation, allowing the user to interrupt the process.

That's a great approach! User is still supervising but not having to do anything but watch, most of the time. That's perfect.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#570

Post by kqr »

diggan wrote: post #9

I still feel like the difference between Sonnet and Opus is a bit unclear. Somewhere on Anthropic's website it says that Opus is the most advanced, but on other parts it says Sonnet is the most advanced and also the fastest. The UI doesn't make the distinction clear either. Then on Perplexity, Perplexity says that Opus is the most advanced, compared to Sonnet. And finally, in the table in the blogpost, Opus isn't eve…

The models "3.5 Sonnet" and "3 Opus" are in my experience nearly at the same level. Once in my last 250 prompts did I run into a problem that 3 Opus was able to solve, but 3.5 Sonnet could not. (I forget the details but it was a combination of logic and trivia knowledge. It is highly likely 3.5 Sonnet would have done a better job with better prompting and richer context, but this was a problem where also I lacked the context and understanding to prompt well.)

Given that 3.5 Sonnet is cheaper and faster than 3 Opus, I default to 3.5 Sonnet so I don't know what the number for the reverse is. How many problems do 3.5 Sonnet get which 3 Opus does not? ¯\_(ツ)_/¯

My best guess would be that it's something in the same kind of range.

Post reply on HN