Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

281–290 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#281

Post by accrual »

gzer0 wrote: post #182

One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…

Seeing models act as though they have agency gives me goosebumps (e.g. seeking out photos of Yellowstone for fun). LLMs don't yet have a concept of true intent or agency, but it's wild to think of them acquiring it.

I have been playing with Mindcraft which lets models interact with Minecraft through the bot API and one of them started saying things like "I want to place some cobblestone there" and then later more general "I want to do X" and then start playing with the available commands, it was pretty cool to watch it explore.

https://github.com/kolbytn/mindcraft

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#282

Post by accrual »

abrichr wrote: post #243
jampekka wrote: post #225

It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.

See https://github.com/OpenAdaptAI/OpenAdapt for an open source alternative that includes operating system accessibility API data and DOM information (along with bitmaps) where available. We are also planning on extracting runtime information using COM/AppleScript: https://github.com/OpenAdaptAI/OpenAdapt/issues/873

It's super cool to see something like this already exists! I wonder if one day something adjacent will become a standard part of major desktop OSs, like a dedicated "AI API" to allow models to connect to the OS, browse the windows and available actions, issue commands, etc. and remove the bitmap parsing altogether as this appears to do.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#283

Post by jjice »

zone411 wrote: post #261

It improves to 25.9 over the previous version of Claude 3.5 Sonnet (24.4) on NYT Connections: https://github.com/lechmazur/nyt-connections/ .

What a neat bench mark! I'm blown away that o1 absolutely crushes everyone else in this. I guess the chain of thought really hashes out those associations.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#284

Post by indigodaddy »

gzer0 wrote: post #182

One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…

This is, craaaaaazzzzzy. I'm just a layman, but to me, this is the most compelling evidence that things are starting to tilt toward AGI that I've ever seen.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#285

Post by _3u10 »

accrual wrote: post #281
gzer0 wrote: post #182

One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…

Seeing models act as though they have agency gives me goosebumps (e.g. seeking out photos of Yellowstone for fun). LLMs don't yet have a concept of true intent or agency, but it's wild to think of them acquiring it. I have been playing with Mindcraft which lets models interact with Minecraft through the bot API and one of them started saying things like "I want to place some cobblestone there" and then later more gen…

What if they do and are just lying to us.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#286

Post by famahar »

KingOfCoders wrote: post #3

I have been a paying ChatGPT customer for a long time (since the very beginning). Last week I've compared ChatGPT to Claude and the results (to my eye) were better, the output better structured and the canvas works better. I'm on the edge of jumping ship.

I'd jump ship if it weren't for the real time voice chat. It's extremely powerful for beginner conversation language learning. Hoping that a company will make use of the real time api for a dedicated language learning app soon.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#287

Post by barrell »

pglevy wrote: post #154
riffraff wrote: post #18

Earlier quoted context omitted.

"computer use" is also as bad a marketing choice as possible for something that actually seems pretty cool.

I had no idea what the headline meant before reading the article. I wasn't even sure how to pronounce "use." (Maybe a typo?) I think something like "Claude adds Keyboard & Mouse Control" would be clearer.

I read the headline 5-10 times trying to make sense of it before even clicking on the link.

Native English speaker, just used the other “use” many times

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#288

Post by paradite »

accrual wrote: post #121
highwaylights wrote: post #81

Completely irrelevant, and it might just be me, but I really like Anthropic's understated branding. OpenAI's branding isn't exactly screaming in your face either, but for something that's generated as much public fear/scaremongering/outrage as LLMs have over the last couple of years, Anthropic's presentation has a much "cosier" veneer to my eyes. This isn't the Skynet Terminator wipe-us-all-out AI, it's the adorable…

I have to agree. I've been chatting with Claude for the first time in a couple days and while it's very on-par with ChatGPT 4o in terms of capability, it has this difficult-to-quantify feeling of being warmer and friendlier to interact with. I think the human name, serif font, system prompt, and tendency to create visuals contributes to this feeling.

Huh. I didn't notice Claude had serif font. Now that I look at it, it's actually mixed. UI elements and user messages are sans serif, chat title and assistant messages are serif.

What an "odd" combination by traditional design standard practices, but surprisingly natural looking on a monitor.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#289

Post by accrual »

submeta wrote: post #272

That’s too much control for my taste. I don’t want anthropic to see my screen. I rather prefer a VS Code with integrated Claude. A version that can see all my dev files in a given folder. I don’t need it to run Chrome for me.

It just depends on the task I suppose. One could have a VM dedicated to a model and let it control it freely to accomplish some set of tasks, then wipe/redeploy if it ever breaks.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#290

Post by accrual »

devinprater wrote: post #208

Maybe LLM's helping blind people like me play video games that aren't accessible to us normally, is getting closer!

Definitely! Those with movement disabilities could have a much easier time if they could just dictate actions to the computer and have them completed with some reliability.
Post reply on HN