Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

731–740 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#731

Post by eperot »

dtquad wrote: post #184
thundergolfer wrote: post #157

Earlier quoted context omitted.

It’s a sandbox compute environment, using Gvisor or Firecracker or similar, which exposes a browser environment to the LLM. modal.com’s modal.Sandbox can be the compute layer for this. It uses Gvisor under the hood.

Is there any Python/Node.js library to easily spawn secure isolated compute environments, possibly using gvisor or firecracker under the hood? This could be useful to build a self-hosted "Computer use" using Ollama and a multimodal model.

I have been [working on one](https://github.com/EtiennePerot/safe-code-execution)! The library is in [src/safecode/sandbox.py](https://github.com/EtiennePerot/safe-code-execution/blob/mas...).

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#732

Post by seany62 »

HarHarVeryFunny wrote: post #466

The "computer use" ability is extremely impressive! This is a lot more than an agent able to use your computer as a tool (and understanding how to do that) - it's basically an autonomous reasoning agent that you can give a goal to, and it will then use reasoning, as well as it's access to your computer, to achieve that goal. Take a look at their demo of using this for coding. https://www.youtube.com/watch?v=vH2f7cjXj…

From what I'm seeing on GH, this could have technically already been built right? Is it not just taking screenshots of the computer screen and deciding what to do from their / looping until it gets to the solution ?

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#733

Post by Eisenstein »

arach wrote: post #729
Eisenstein wrote: post #687

Earlier quoted context omitted.

> if AI gets even just 2 orders of magnitude better over the next two years You realize this means '100 times better', right?

yes, thanks for pointing out the assumption here. I'm not sure how to quantify AI improvements and tbh not really up to speed on quantifiable rate of improvement from 4 to 4o to o1 100 times better seems to me in line with the bet that's justifying $250B per annum in Cap Ex (just among hyperscalers) but curious how you might project a few years out? Having said that, my use of 100x better here applies to 100x more ef…

I would think that such an increase in AI capability would basically be AGI...

Just to give a few comparisons, the following things are two orders of magnitude apart:

1. The force felt by a mosquito landing on your arm and getting punched by Mike Tyson in his prime

2. A firecracker exploding and a stick of dynamite exploding

3. The heat from a candle and the heat from a blowtorch

4. The sound from a whisper and the sound from jet engine

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#734

Post by saaaaaam »

runako wrote: post #510
mathnmusic wrote: post #473

Earlier quoted context omitted.

It does not have access to the Excel app. You may be able to generate the .xlsx file using python libraries but you would need to run the python code on your own. ChatGPT can run generated code, which is probably why it works there.

I don't expect it to have the Excel app, I expect it to run the code it is capable of generating. This is what I mean by their strategy being a jumble. Claude can do the hard part of figuring out what code to write and writing it, but then refuses to do the easier part of executing the code.

See, I think this is a case of personal preference. I much prefer Claude's approach of figuring out the code and writing it for me to execute myself, rather than having it all in one box. Apart from anything else, it makes me think a little more about the process, and the desired outcome, rather than just iterate, click, iterate, click.

It's marginally less efficient, for sure, but it allows me greater visibility on the process, and gives me more confidence that whatever it's doing is what I want it to do.

But maybe that's some weird luddite-ism on my part, and I should just embrace an even blacker box where everything is done in the tool.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#735

Post by Daniel_Van_Zant »

girvo wrote: post #419
tkellogg wrote: post #410

Earlier quoted context omitted.

Honestly, this is going to be huge for healthcare. There's an incredible amount of waste due to incumbent tech making interoperability difficult.

We’ll see. Having worked in this space in the past, the technical challenges are able to overcome today with no new technology: its a business sales and regulation challenge more than a tech one.

I agree that at the business contract level, it's more about sales and regulations than tech. But in my experience working close to minimum wage white-collar jobs, about 1 in 4 of my coworkers had automated most of their job with some unholy combination of VBScript, Excel wizardry, AutoHotKey, Selenium, and just a bit of basic Python sprinkled in; IT, security, and privacy concerns notwithstanding. Some were even dedicated enough to pay small amounts out-of-pocket for certain tools.

I'd bet that until we get the risks whittled down enough for larger organizations to adopt this on a wide scale, the biggest user group for AI automation tools will be at the level of individual workers who are eager to streamline their own tasks and aren't paid enough to care about those same risks.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#736

Post by Daniel_Van_Zant »

sdwr wrote: post #434
LASR wrote: post #372

Earlier quoted context omitted.

That's exactly it. I've been peddling my vision of "AI automation" for the last several months to acquaintances of mine in various professional fields. In some cases, even building up prototypes and real-user testing. Invariably, none have really stuck. This is not a technical problem that requires a technical solution. The problem is that it requires human behavior change. In the context of AI automation, the promis…

There's a huge huge gap between "coaxing what you want out of it" and "trusting it to perform flawlessly". Everybody on the planet would use #2, but #1 is just for enthusiasts. AI is squarely #1. You can't trust it with your credit card to order groceries, or to budget and plan and book your vacation. People aren't picking up on AI because it isn't good enough yet to trust - you still have the burden of responsibilit…

I think there is a large pool of near minimum-wage white collar workers who wouldn't care about that difference when it comes to executing on their jobs. These are the folks who are already using VBScript, AutoHotKey, Excel wizardry, etc. to automate large parts of their job regardless of any risks and will continue to use these new tools for similar purposes.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#737

Post by HarHarVeryFunny »

seany62 wrote: post #732
HarHarVeryFunny wrote: post #466

The "computer use" ability is extremely impressive! This is a lot more than an agent able to use your computer as a tool (and understanding how to do that) - it's basically an autonomous reasoning agent that you can give a goal to, and it will then use reasoning, as well as it's access to your computer, to achieve that goal. Take a look at their demo of using this for coding. https://www.youtube.com/watch?v=vH2f7cjXj…

From what I'm seeing on GH, this could have technically already been built right? Is it not just taking screenshots of the computer screen and deciding what to do from their / looping until it gets to the solution ?

Well, obviously it's controlling your computer too - controlling mouse and keyboard input, and has been trained to know how to interact with apps (how to recognize and use UI components). It's not clear exactly what all the moving parts are and how they interact.

I wouldn't be so dismissive - you could describe GPT-o1 in same way "it just loops until it gets to the solution". It's the details and implementation that matter.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#738

Post by runako »

saaaaaam wrote: post #734
runako wrote: post #510

Earlier quoted context omitted.

I don't expect it to have the Excel app, I expect it to run the code it is capable of generating. This is what I mean by their strategy being a jumble. Claude can do the hard part of figuring out what code to write and writing it, but then refuses to do the easier part of executing the code.

See, I think this is a case of personal preference. I much prefer Claude's approach of figuring out the code and writing it for me to execute myself, rather than having it all in one box. Apart from anything else, it makes me think a little more about the process, and the desired outcome, rather than just iterate, click, iterate, click. It's marginally less efficient, for sure, but it allows me greater visibility on…

YMMV obviously. If I ask the magic box to make a spreadsheet, I don't need to see the Python for that any more than I need to see the code it uses to summarize something I paste in. I don't really even care that it has to write code to make the spreadsheet at all.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#739

Post by runako »

sumedh wrote: post #656
runako wrote: post #369

I really don't get their model. They have very advanced models, but the service overall seems to be a jumble of priorities. Some examples: Anthropic doesn't offer an unlimited chatbot service, only plans that give you "more" usage, whatever that means. If you have an API key, you are "unlimited," so they have the capability. Why doesn't the chatbot allow one to use their API key in the Claude app to get unlimited usa…

> Anthropic doesn't offer an unlimited chatbot service, because its expensive, if they give unlimited service someone will misuse it

You misunderstood. Anthropic does not offer a paid Claude offering akin to ChatGPT Plus, where usage is effectively unlimited.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#740

Post by bamboozled »

lobochrome wrote: post #567
csomar wrote: post #561

I like its lazy approach > I apologize, but it seems we're having some issues installing Chromium. As an alternative, since I can see Firefox is already installed on the system, would you like to use Firefox instead? Or would you prefer me to try a different approach to installing Chromium? It seems unable to figure out how to install chormium. Sees the Firefox icon out there and just want to use it. You know, like a…

SYSTEM_PROMPT = f""" * You are utilising an Ubuntu virtual machine using {platform.machine()} architecture with internet access. * You can feel free to install Ubuntu applications with your bash tool. Use curl instead of wget. * To open firefox, please just click on the firefox icon. Note, firefox-esr is what is installed on your system. * Using bash tool you can start GUI applications, but you need to set export DIS…

Quite amazing we can program a computer with natural language like this.
Post reply on HN