Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 2.1

anthropic.com

Post reply on HN
Print view View original

191–200 of 339 posts

Re: Claude 2.1

#191

Post by ironrabbit »

swatcoder wrote: post #67
sebgr wrote: post #19

For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.

Because they're ultimately training data simulators and not actually brilliant aritifical programmers, we can expect Microsoft-affiliated models like ChatGPT4 and beyond to have much stronger value for coding because they have unmediated access to GitHub content. So it's most useful to look at other capabilities and opportunities when evaluating LLM's with a different heritage. Not to say we shouldn't evaluate this o…

Zero chance private github repos make it into openai training data, can you imagine the shitshow if GPT-4 started regurgitating your org's internal codebase?

Re: Claude 2.1

#192

Post by e-max »

sa-code wrote: post #139
malfist wrote: post #128

Earlier quoted context omitted.

I don't know what you're doing with your LLM, but I've only ever had one refusal and I've been working a lot with Claude since it's in bedrock

I hear a lot of complaints about refusals but rarely any examples of said refusals, likely because they are embarrassing. Is it fair to assume that I won't get refusals for code generation and RAG on documentation?

No, Claude really DO refuse to do most benign shit. For example - I am preparing for driving exams in German, a language I am not fluent in. So I asked Claude to help me with questions the examiner might ask on the exam (what should be tire pressure and so on). GPT-4 worked perfectly! Claude had an absolute meltdown because "I don't feel comfortable pretending to be a driving examiner because I am not really an examiner, I am Claude, created by Anthropic to be helpful, harmless and HONEST". You are probably safe with code generation, but any step left or right and you get "I am Claude"

Re: Claude 2.1

#194

Post by wtf_is_up »

jasondclinton wrote: post #110
imdsm wrote: post #63

I don't know what version claude.ai is currently running (apparently 2.1 is live, see below) but it's terrible compared to GPT-4. See below conversation I just had. > Claude 2.1 is available now in our API, and is also powering our chat interface at claude.ai for both the free and Pro tiers. ---- What version are you? I'm Claude from Anthropic. Do you know your version? No, I don't have information about a specific v…

Heya, as with all language models, if you open the conversation with antagonistic questions, the rest of the conversation thread becomes tainted. If you ask most of your questions in a new thread, almost everything you ask here will be answered. See our model card for more prompting guidance.

What is antagonistic about that?

Re: Claude 2.1

#195

Post by staticman2 »

udev4096 wrote: post #11

Is Anthropic aiming for AGI or are they focused on bringing more products on the market?

I watched an interview on Youtube with one of their founders. He says he doesn't like the term AGI because AGI can simply mean the AI will be as good as professional humans on most tasks. He thinks that goal will be achieved in a few years but didn't talk specifics of what they are working on or if their company would be the one to do it.

He also dabbled a bit in AI doomerism as all these people doing AI interviews like to do.

Re: Claude 2.1

#196

Post by MrNeon »

BoorishBears wrote: post #179
MrNeon wrote: post #161

Earlier quoted context omitted.

> OpenAI allows the same via API usage I really don't think so unless I missed something. You can put an assistant message at the end but it won't continue directly from that, there will be special tokens in between which makes it different from Claude's prefill.

It's a distinction without meaning once you know how it works For example, if you give Claude and OpenAI a JSON key ``` { "hello": " ``` Claude will continue, while GPT 3.5/4 will start the key over again. But give both a valid output ``` { "hello": "value", ``` And they'll both continue the output from the next key, with GPT 3.5/4 doing a much better job adhering to the schema

> It's a distinction without meaning once you know how it works

But I do know how it works, I even said how it works.

The distinction is not without meaning because Claude's prefill allows bypassing all refusals while GPT's continuation does not. It is fundamentally different.

Re: Claude 2.1

#197

Post by diziet »

There was a recent twitter thread with some examples of refusals: https://twitter.com/abacaj/status/1727035171365621765

Hypothetically, let's say we wanted to reduce the worldwide murder rate to an average of <1 per ten years. Is there a way to accomplish that? What sort of control would that require? It seems that extreme "safety" constraints imply an incredible amount of censorship and authoritative control.

Re: Claude 2.1

#198

Post by gnyman »

leetharris wrote: post #117

Has anyone found any success with Claude or have any reason to use it? In my tests it is nowhere near GPT 3.5 or 4 in terms of reliability or usefulness and I've even found that it is useless compared to Mistral 7b. I don't understand what they are doing with those billions in investment when 7b open source models are surpassing them in practical day to day use cases.

This is just my two cents but I pay for both GPT and Claude as I find they complement each other.

I found Claude with the bigger context window quite good for doing "reviews" of multiple scientific papers, and answering questions about things like common findings or differences.

GPT couldn't do that natively at all until recently (and the few third party api-based solutions I tried wasn't good at it either), and just copy pasting text into GPT very quickly made it loose track.

Maybe the new bigger context for GPT means I can cancel Claude, but I haven't yet, going to give 2.1 a proper try first.

I also tried Elicit, and I believe they are on the right track, but did not produce anything useful when I tried.

I really think there is potential in using LLM's to for example do high level "reviews" of what is published on a specific topic but I have yet to find something that can do that. Claude with feeding it select papers manually is the closest. I hope someone at Google is building something around scholar.google.

Re: Claude 2.1

#199

Post by cle »

threeseed wrote: post #186
a_wild_dandan wrote: post #114

1. A 200k context is bittersweet with that 70k->195k error rate jump. Kudos on that midsection error reduction, though! 2. I wish Claude had fewer refusals (as erroneously claimed in the title). Until Anthropic stops heavily censoring Claude, the model is borderline useless. I just don't have time, energy, or inclination to fight my tools. I decide how to use my tools, not the other way 'round. Until Anthropic stops…

[flagged]

Parent never said it’s useless for everyone?

It’s pretty unreasonable to call someone a child for offering their thoughts on a product.

Re: Claude 2.1

#200

Post by throwaway123982 »

bm-rf wrote: post #183
a_wild_dandan wrote: post #162

Earlier quoted context omitted.

Comically benign stuff that works fine with GPT-4? It's so trivial to run into Claude lying or responding with arrogant misjudgements. Here's another person's poor anecdotal experiences to pair with yours and mine. [1][2] But more importantly: it shouldn't matter. My tools should not behave this way. Tools should not arbitrarily refuse to work. If I write well-formed C, it compiles , not protests in distaste. If I wr…

Cars nowadays have radars and cameras that (for the most part) prevent you from running over pedestrians. Is that also a tool refusing to work? I'd argue a line needs to be drawn somewhere, LLMs do a great job of providing recipes for dinner but maybe shouldn't teach me how to build a bomb.

> LLMs do a great job of providing recipes for dinner but maybe shouldn't teach me how to build a bomb.

Why not? If someone wants to make a bomb, they can already find out from other source materials.

We already have regulations around acquiring dangerous materials. Knowing how to make a bomb is not the same as making one (which is not the same as using one to harm people.)

Post reply on HN