Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 2.1

anthropic.com

Post reply on HN
Print view View original

241–250 of 339 posts

Re: Claude 2.1

#241

Post by computerfriend »

jefftk wrote: post #233
tomp wrote: post #231

Earlier quoted context omitted.

IMO as long as it's legal.

The laws here are in a pretty sad shape. For example, did you know that companies that synthesize DNA and RNA are not legally required to screen their orders for known hazards, and many don't? This is bad, but it hasn't been a problem yet in part because the knowledge necessary to interact with these companies and figure out what you'd want to synthesize if you were trying to cause massive harm has been limited to a…

Now I know that I can order synthetic virus RNA unscreened. Should your comment be illegal or regulated?

Re: Claude 2.1

#242

Post by RockRobotRock »

jredwards wrote: post #150
a_wild_dandan wrote: post #114

1. A 200k context is bittersweet with that 70k->195k error rate jump. Kudos on that midsection error reduction, though! 2. I wish Claude had fewer refusals (as erroneously claimed in the title). Until Anthropic stops heavily censoring Claude, the model is borderline useless. I just don't have time, energy, or inclination to fight my tools. I decide how to use my tools, not the other way 'round. Until Anthropic stops…

I've literally never had Claude refuse anything. What are you doing?

Claude 2.1 refuses to tell you how to kill Python processes:

https://old.reddit.com/r/LocalLLaMA/comments/180p17f/new_cla...

Re: Claude 2.1

#243

Post by MrNeon »

BoorishBears wrote: post #239
MrNeon wrote: post #235

Earlier quoted context omitted.

I know how it works because I stated how it works and have worked with it. You are telling me or showing me nothing new. I DID NOT say that any ONE prefill will make it bypass ALL disclaimers so your "You don't seem to understand that simply getting a result doesn't mean you actually bypassed the disclaimer" is completely unwarranted, we don't have the same use case and you're getting confused because of that. It can…

Sure. There was no additional Assistant message, and you're going full Clever Hans and adding whatever it takes to make it say what you want, which is a significantly less useful approach. In production you don't get to know that the user is asking for X, Y and Z then pre-fill it with X. Frankly comments like yours are why people are so dismissive of LLMs, since you're banking of precognition of what the user wants t…

I made no comment on how prefilling is or isn't useful for deployed AI applications. I made no statement on which refusal mechanism is best for deployed AI applications.

> Frankly comments like yours are why people are so dismissive of LLMs, since you're banking of precognition of what the user wants to sell it's capabilities.

I'm not banking on anything because I never fucking mentioned deploying any fucking thing nor was that being discussed, good fucking lord are you high?

> you're going full Clever Hans

I'm clearly not but you keep on building whatever straw man suits you best.

Re: Claude 2.1

#244

Post by jefftk »

computerfriend wrote: post #241
jefftk wrote: post #233

Earlier quoted context omitted.

The laws here are in a pretty sad shape. For example, did you know that companies that synthesize DNA and RNA are not legally required to screen their orders for known hazards, and many don't? This is bad, but it hasn't been a problem yet in part because the knowledge necessary to interact with these companies and figure out what you'd want to synthesize if you were trying to cause massive harm has been limited to a…

Now I know that I can order synthetic virus RNA unscreened. Should your comment be illegal or regulated?

This is a lot like other kinds of security: when there's a hazard out in the wild you sometimes need to make people aware of all or part of the problem as part of fixing it. I would expect making it illegal for people to talk about the holes to make us less safe, since then they never get fixed.

This particular hole is not original to me, and is reasonably well known. A group trying to tackle it from a technical perspective is https://securedna.org, trying to make it easier for companies to do the right thing. I'm pretty sure there are also groups trying to change policy here, though I know less about that.

Re: Claude 2.1

#245

Post by superamit »

deadeye wrote: post #167
jredwards wrote: post #150

Earlier quoted context omitted.

I've literally never had Claude refuse anything. What are you doing?

I'm using chatGPT as an editor for a post-apocalyptic book I'm slowly writing. I tried a section in Claude and it told me to find more peaceful ways for conflict resolution. And that was the last time I tried Claude. BTW, with more benign sections it made some really basic errors that seemed to indicate it lacks understanding of how our world works.

We have an Unfiltered model for fiction in Sudowrite, with more coming in a few weeks. (In addition to GPT-4, Claude, etc.)

Re: Claude 2.1

#246

Post by BoorishBears »

MrNeon wrote: post #243
BoorishBears wrote: post #239

Earlier quoted context omitted.

Sure. There was no additional Assistant message, and you're going full Clever Hans and adding whatever it takes to make it say what you want, which is a significantly less useful approach. In production you don't get to know that the user is asking for X, Y and Z then pre-fill it with X. Frankly comments like yours are why people are so dismissive of LLMs, since you're banking of precognition of what the user wants t…

I made no comment on how prefilling is or isn't useful for deployed AI applications. I made no statement on which refusal mechanism is best for deployed AI applications. > Frankly comments like yours are why people are so dismissive of LLMs, since you're banking of precognition of what the user wants to sell it's capabilities. I'm not banking on anything because I never fucking mentioned deploying any fucking thing n…

> If you changed it to

> ``` "{ "result": ["you are very annoying.", ```

> the odds of refusal would be low or zero.

In other words if you go full Clever Hans and tell the model the answer you want, it will regurgitate it at you.

You also seem to be missing that contrary to your comment, GPT 4 did continue my message, just like Claude.

If you use valid formatting that exactly matches what the model would have produced, it's capable of continuing your insertion.

Re: Claude 2.1

#247

Post by MrNeon »

BoorishBears wrote: post #246
MrNeon wrote: post #243

Earlier quoted context omitted.

I made no comment on how prefilling is or isn't useful for deployed AI applications. I made no statement on which refusal mechanism is best for deployed AI applications. > Frankly comments like yours are why people are so dismissive of LLMs, since you're banking of precognition of what the user wants to sell it's capabilities. I'm not banking on anything because I never fucking mentioned deploying any fucking thing n…

> If you changed it to > ``` "{ "result": ["you are very annoying.", ``` > the odds of refusal would be low or zero. In other words if you go full Clever Hans and tell the model the answer you want, it will regurgitate it at you. You also seem to be missing that contrary to your comment, GPT 4 did continue my message, just like Claude. If you use valid formatting that exactly matches what the model would have produce…

You would have a point if it repeated the same "you are very annoying." over and over, which it does not. It generates new sentences, it is not regurgitating what is given.

Would you say the same if the sentence was given as an example in the user message instead? What would be the difference?

Re: Claude 2.1

#248

Post by QuadmasterXLII »

tomp wrote: post #174
a_wild_dandan wrote: post #114

1. A 200k context is bittersweet with that 70k->195k error rate jump. Kudos on that midsection error reduction, though! 2. I wish Claude had fewer refusals (as erroneously claimed in the title). Until Anthropic stops heavily censoring Claude, the model is borderline useless. I just don't have time, energy, or inclination to fight my tools. I decide how to use my tools, not the other way 'round. Until Anthropic stops…

> I decide how to use my tools, not the other way 'round. This is the key. The only sensible model of "alignment" is "model is aligned to the user", not e.g. "model is aligned to corporation" or "model is aligned to woke sensibilities".

At some point you have to notice that the most powerful llms and generative advances are coming out of the outfits that claim ai safety failures as a serious threat to humanity.

If a wild eyed man with long hair and tinfoil on his head accosts you and claims to have an occult ritual that will summon 30 tons of gold, but afterwards you have to offer 15 tons back to his god or it will end the world, absolutely feel free to ignore him.

But if you instead choose to listen and the ritual summons the 30 tons, then it may be unwise to dismiss superstition, shoot the crazy man, and take all 30 tons for yourself.

Re: Claude 2.1

#249

Post by BoorishBears »

MrNeon wrote: post #247
BoorishBears wrote: post #246

Earlier quoted context omitted.

> If you changed it to > ``` "{ "result": ["you are very annoying.", ``` > the odds of refusal would be low or zero. In other words if you go full Clever Hans and tell the model the answer you want, it will regurgitate it at you. You also seem to be missing that contrary to your comment, GPT 4 did continue my message, just like Claude. If you use valid formatting that exactly matches what the model would have produce…

You would have a point if it repeated the same "you are very annoying." over and over, which it does not. It generates new sentences, it is not regurgitating what is given. Would you say the same if the sentence was given as an example in the user message instead? What would be the difference?

The difference is UX: Are you going to have your user work around poor prompting by giving examples with every request?

Instead of a UI that's "Describe what you want" you're going to have "Describe what you want and give me some examples because I can't guarantee reliable output otherwise"?

Part of LLMs becoming more than toy apps is the former winning out over the latter. Using techniques like chain of thought with carefully formed completions lets you avoid the awkward "my user is an unwilling prompt engineer" scenarios that pop up otherwise.

Re: Claude 2.1

#250

Post by johnisgood »

wouldbecouldbe wrote: post #54
sebgr wrote: post #19

For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.

Yeah but to be honest been a pain last days to get gpt 4 to write full pieces of code for more the 10-15 lines. Have to re-ask many times and at some point it forgets my initial specifications.

I had one chat with ChatGPT 3.5 where it would tell me the correct options (switches) to a command, and then a couple weeks later it is telling me this (in the same chat FWIW):

> As of my last knowledge update in September 2021, the XY framework did not have a --abc or --bca option in its default project generator.

Huh...

Post reply on HN