Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

anthropic.com

Post reply on HN
Print view View original

591–600 of 758 posts

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#591

Post by dygd »

itchyjunk wrote: post #577
lobochrome wrote: post #567

Earlier quoted context omitted.

SYSTEM_PROMPT = f""" * You are utilising an Ubuntu virtual machine using {platform.machine()} architecture with internet access. * You can feel free to install Ubuntu applications with your bash tool. Use curl instead of wget. * To open firefox, please just click on the firefox icon. Note, firefox-esr is what is installed on your system. * Using bash tool you can start GUI applications, but you need to set export DIS…

Is this you guessing the system prompt? Is this the actual prompt the model has? What is the context?

It appears to be the system prompt of the quickstart app: https://github.com/anthropics/anthropic-quickstarts/blob/mai...

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#592

Post by maeil »

hobofan wrote: post #189
dotancohen wrote: post #135

Earlier quoted context omitted.

> B) potentially phase out Opus, and instead introduce new branding for what they called a "reasoning model" like OpenAI did with o1(-preview) When should we be using the -o OpenAI models? I've not been keeping up and the official information now assumes far too much familiarity to be of much use.

I think it's first important to note that there is a huge difference between -o models (GPT 4o; GPT 4o mini) and the o1 models (o1-preview; o1-mini). The -o models are "just" stronger versions of their non-suffixed predecessors. They are the latest (and maybe last?) version of models in the lineage of GPT models (roughly GPT-1 -> GPT-2 -> GPT-3 -> GPT-3.5 -> GPT-4 -> GPT-4o). The o1 models (not sure what the naming s…

> The -o models are "just" stronger versions of their non-suffixed predecessors.

Cheaper and faster, but not notably "stronger" at real-world use.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#593

Post by vramana »

codebolt wrote: post #586
seriocomic wrote: post #580

Earlier quoted context omitted.

What's the (current) best way to integrate it? VS Code extension? Other IDE?

I'll throw this out here as well: Is there any decent alternative to GitHub Copilot when using Visual Studio? (Pretty happy with it to be fair, but would be open to trying others.)

Supermaven is really good. I am a paying user of super maven.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#594

Post by Squarex »

seriocomic wrote: post #580
stavros wrote: post #491

Earlier quoted context omitted.

I use Claude for coding and it's fantastic. I definitely have outsourced a lot of my coding to it.

What's the (current) best way to integrate it? VS Code extension? Other IDE?

Sourcegraph Cody.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#595

Post by TechDebtDevin »

machiaweliczny wrote: post #457
TechDebtDevin wrote: post #263

Not that I'm scared of this update but I'd probably be alright with pausing llm development today, atleast in regard to producing code. I don't want an llm to write all my code, regardless of if it works, I like to write code. What these models are capable of at the moment is perfect for my needs and I'd be 100% okay if they didn't improve at all going forward. Edit: also I don't see how an llm controlled system can…

In normal critical system u use 3 CPUs. With LLM u can 1000 shot majority voting. Seems like approaches like entropix might reduce hallucinations also.

I don't think this snapshot image/vision model is going to be the best solution. I think CLI is a much better interface for llms.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#596

Post by Vampiero »

zeroonetwothree wrote: post #445
ianeigorndua wrote: post #431

Earlier quoted context omitted.

Are these synthetic or real-world benchmarks? Answering myself: ”Aider’s code editing benchmark asks the LLM to edit python source files to complete 133 small coding exercises from Exercism” Not gonna start looking for a job any time soon

Example I chose at random: > Convert a hexadecimal number, represented as a string (e.g. "10af8c"), to its decimal equivalent using first principles (i.e. no, you may not use built-in or external libraries to accomplish the conversion). So it's fairly synthetic. It's also the sort of thing LLMs should be great at since I'm sure there's tons of data on this sort of thing online.

Yeah but programming isn't about solving problems that were solved millions of times already. I mean, web dev kind of is, but that's not the point. If a problem is solved, then it's just a matter of implementing the solution and anyone can do that given the proper instructions (even without understanding how or why they solve the problem).

I've formalized a lot of stuff I didn't understand just by copying the formulas from Wikipedia.

As long as LLMs are not capable of proper reasoning, they will remain a gimmick in the context of programming.

They should really just focus on refactoring benchmarks across many languages. If an AI can refactor my complex code properly without changing the semantics, it's good enough for me. But that unfortunately requires such a high-level understanding of the codebase that with the current tech it's just impossible to get a half-decent result in any real-world scenario.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#597

Post by kortilla »

WalterSear wrote: post #411
unshavedyak wrote: post #364

Earlier quoted context omitted.

I feel pain for the people who will be employed to "prompt engineer" the behavior of these things. When they inevitably hallucinate some insane behavior a human will have to take blame for why it's not working.. and yea, that'll be fun to be on the receiving end of.

Humans 'hallucinate' like LLMs. The term used however, is confabulation: we all do it, we all do it quite frequently, and the process is well studied(1). > We are shockingly ignorant of the causes of our own behavior. The explanations that we provide are sometimes wholly fabricated, and certainly never complete. Yet, that is not how it feels. Instead it feels like we know exactly what we're doing and why. This is con…

No, confabulation isn’t anything like how LLMs hallucinate. LLMs will just very confidently make up APIs on systems they otherwise clearly have been trained on.

This happens nearly every time I request “how tos” for libraries that aren’t very popular. It will make up some parameters that don’t exist despite the rest of the code being valid. It’s not a memory error like confabulation where it’s convinced the response is valid from memory either, because it can be easily convinced that it made a mistake.

I’ve never worked with an engineer in my 25 years in the industry that has done this. People don’t confabulate to get day to day answers. What we call hallucination is the exact same process LLMs use to get valid answers.

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#599

Post by aryehof »

tammer wrote: post #330

This demo is impressive although my initial reaction is a sort of grief that I wasn't born in the timeline where Alan Kay's vision of object-oriented computing was fully realized -- then we wouldn't have to manually reconcile wildly heterogeneous data formats and interfaces in the first place!

Vision of a “universal communicator” rather than OO?

Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku

#600

Post by spaceman_2020 »

stavros wrote: post #491
ianeigorndua wrote: post #431

Earlier quoted context omitted.

Are these synthetic or real-world benchmarks? Answering myself: ”Aider’s code editing benchmark asks the LLM to edit python source files to complete 133 small coding exercises from Exercism” Not gonna start looking for a job any time soon

I use Claude for coding and it's fantastic. I definitely have outsourced a lot of my coding to it.

The new version is already in Cursor and its outstanding.

Can code at mid-level now. Almost.

Post reply on HN