Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

741–750 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#741

Post by WalterSear »

kortilla wrote: post #597
WalterSear wrote: post #411

Earlier quoted context omitted.

Humans 'hallucinate' like LLMs. The term used however, is confabulation: we all do it, we all do it quite frequently, and the process is well studied(1). > We are shockingly ignorant of the causes of our own behavior. The explanations that we provide are sometimes wholly fabricated, and certainly never complete. Yet, that is not how it feels. Instead it feels like we know exactly what we're doing and why. This is con…

No, confabulation isn’t anything like how LLMs hallucinate. LLMs will just very confidently make up APIs on systems they otherwise clearly have been trained on. This happens nearly every time I request “how tos” for libraries that aren’t very popular. It will make up some parameters that don’t exist despite the rest of the code being valid. It’s not a memory error like confabulation where it’s convinced the response…

You work with engineers who confabulate all the time: it's an intrinsic aspect of how the human brain functions that has been demonstrated at multiple levels of cognition.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#742

Post by jviotti »

nbzso wrote: post #621

Just a question: For this thingy to work, I must give the provider access to my computer? Good luck. :) Just another reason to use ONLY local LLM's.

This. There is no way I would trust any AI provider to pretty much have full control over my computer.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#743

Post by aoeusnth1 »

a9dhalaan wrote: post #690
aoeusnth1 wrote: post #368

Earlier quoted context omitted.

Yes, (old) 3.5 Sonnet is distinctly worse at emotional intelligence, flexibility, expressiveness and poetry.

Are you also implying that new 3.5 sonnet is better at those things?

No, Opus is better. I have no experience with 3.5.new.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#744

Post by snowe2010 »

weweweoo wrote: post #615
snowe2010 wrote: post #588

Earlier quoted context omitted.

Defining AGI as “can reason about 5MLOC” is ridiculous. When do the goal posts stop moving? When a computer can solve time travel? Babies have behavior all the time that is no more differentiable from what an LLM does on a normal basis (including terrible logic and hallucinations). The majority of people on the planet can barely reason about how any given politician will affect them, even when there’s a billion resou…

Babies can at least manipulate the physical world. Large language model can never be defined as AGI until it can control a general purpose robot, similar to how human brain controls our body's motor functions.

You’re commenting that on an article about how Claude literally can do what you’re talking about.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#745

Post by snowe2010 »

aerhardt wrote: post #682
snowe2010 wrote: post #588

Earlier quoted context omitted.

Defining AGI as “can reason about 5MLOC” is ridiculous. When do the goal posts stop moving? When a computer can solve time travel? Babies have behavior all the time that is no more differentiable from what an LLM does on a normal basis (including terrible logic and hallucinations). The majority of people on the planet can barely reason about how any given politician will affect them, even when there’s a billion resou…

As generally intelligent beings, we can adapt to reading and producing 5M LOC, or to live in arctic climates, or to build a building in colonial or classical style as dictated by cost, taste, and other factors. That is generality in intelligence. I haven't moved any goal posts - it is your definition which is way too narrow.

You’re literally moving the goalposts right now. These models _are_ adapting to what you’re talking about. When Claude makes a model for haikus, how is that different than a poet who knows literally nothing about math but is fantastic at poetry?

I’m sure as soon as Claude can handle 5MLOC you’ll say it should be 10, and it needs to make sure it can serve you a Michelin star dinner as well.

That’s not AGI. Stop moving the goalposts.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#746

Post by sumedh »

runako wrote: post #739
sumedh wrote: post #656

Earlier quoted context omitted.

> Anthropic doesn't offer an unlimited chatbot service, because its expensive, if they give unlimited service someone will misuse it

You misunderstood. Anthropic does not offer a paid Claude offering akin to ChatGPT Plus, where usage is effectively unlimited.

OpenAI has more funding and more customers than Anthropic.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#749

Post by distalx »

marsh_mellow wrote: post #19

Anthropic blog post outlining the research process: https://www.anthropic.com/news/developing-computer-use Computer use API documentation: https://docs.anthropic.com/en/docs/build-with-claude/compute... Computer Use Demo: https://github.com/anthropics/anthropic-quickstarts/tree/mai...

On their "Developing a computer use model" post they have mention > On one evaluation created to test developers’ attempts to have models use computers, OSWorld, Claude currently gets 14.9%. That’s nowhere near human-level skill (which is generally 70-75%), but it’s far higher than the 7.7% obtained by the next-best AI model in the same category.

Here, "next-best AI model in the same category" referes to which model.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#750

Post by rubslopes »

ssijak wrote: post #249
rubslopes wrote: post #205

Earlier quoted context omitted.

I've been using a lot of o1-mini and having a good experience with it. Yesterday I decided to try sonnet 3.5. I asked for a simple but efficient script to perform fuzzy match in strings with Python. Strangely, it didn't even mention existing fast libraries, like FuzzyWuzzy and Rapidfuzz. It went on to create everything from scratch using standard libraries. I don't know, I thought this was something basic for it to s…

just ask it to use libraries you want; you cant expect it to magically read your mind, you need to guide every LLM to what are your must/nice haves

Hey, I just wanted to say that I kept using it and the new Sonnet 3.5 is so much better than any other competitor.
Post reply on HN