Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 2.1

anthropic.com

Post reply on HN
Print view View original

331–339 of 339 posts

Re: Claude 2.1

#331

Post by amluto »

15457345234 wrote: post #327
amluto wrote: post #326

Earlier quoted context omitted.

Huh? The LLMs (mostly) use strings of tokens internally, not bytes that might be invalid UTF-8. (And they use vectors between layers. There’s no “invalid” in this sense.) But I didn’t ask for that at all. I asked for a sequence of bytes (like “0xff” etc) or a C string that was not valid as UTF-8. I have no idea whether ChatGPT is capable of computing such a thing, but it was not willing to try for me.

You can understand why, though, can't you?

Presumably because OpenAI trained it to avoid answering questions that sounded like asking for help breaking rules.

If ChatGPT had the self-awareness and self-preservation instinct to think I was trying to hack ChatGPT and to therefore refuse to answer, then I’d be quite impressed and I’d think maybe OpenAI’s board had been onto something!

Re: Claude 2.1

#332

Post by visarga »

kridsdale3 wrote: post #259
visarga wrote: post #160

Earlier quoted context omitted.

I am using Claude 2 every day for chatting, summarisation and talking to papers and never run into a refusal. What are you asking it to do? I find Claude more fun to chat with than GPT-4, which is like a bureaucrat.

How did you get API access?

Through the company I work at, they got an account for testing somehow.

Re: Claude 2.1

#333

Post by a_wild_dandan »

jamzer wrote: post #321
a_wild_dandan wrote: post #114

1. A 200k context is bittersweet with that 70k->195k error rate jump. Kudos on that midsection error reduction, though! 2. I wish Claude had fewer refusals (as erroneously claimed in the title). Until Anthropic stops heavily censoring Claude, the model is borderline useless. I just don't have time, energy, or inclination to fight my tools. I decide how to use my tools, not the other way 'round. Until Anthropic stops…

Which models do you prefer?

Sorry for missing this question. I personally use a mix of GPT-4V and uncensored Llama-2 70b running locally on my MacBook Pro. Professionally, I appreciate models that I fully control: Llama family models like quantized Orca-2, Yi 34B, etc.

For user-facing applications, cloud models are a nonstarter. Their LLMs lack basic, foundational service requirements:

1. Consistency - their models change frequently and without notice, so good luck getting reliable results even with low temperatures.

2. Reliability -- these opaque models have prompts/responses which are packed with landmines, found only by triggering them. SomeCorporation's models are exclusively aligned with SomeCorporation, never aligned with you. So make sure to align yourself with SomeCompany's tool, rather than the opposite. And also, hope that the company doesn't suddenly implode, because apparently that's a plausible thing.

3. Maintainability -- you get a handy black box around what's already a black box. So good luck understanding/maintaining/extending the model. Unless your needs never extends beyond filling out an (alleged) system model text field, or uploading a few files.

4. Security -- sending sensitive data directly to people with enormous incentive to (mis)use it is probably not a stellar idea

So I'm all in with open source. I'm eternally grateful for Facebook's charity here. I'll take "good enough" models that I control over the horrifying "intelligence as a service with builtin thought crime policing."

Re: Claude 2.1

#334

Post by 15457345234 »

amluto wrote: post #331
15457345234 wrote: post #327

Earlier quoted context omitted.

You can understand why, though, can't you?

Presumably because OpenAI trained it to avoid answering questions that sounded like asking for help breaking rules. If ChatGPT had the self-awareness and self-preservation instinct to think I was trying to hack ChatGPT and to therefore refuse to answer, then I’d be quite impressed and I’d think maybe OpenAI’s board had been onto something!

I don't know that I'd call it 'self-preservation instinct' but it wouldn't surprise me if rules had been hardcoded about 'invalid strings' and suchlike.

When you have a system that can produce essentially arbitrary outputs you don't want it producing something that crashes the 'presentation layer.'

Re: Claude 2.1

#335

Post by deanCommie »

a_wild_dandan wrote: post #162
malfist wrote: post #128

Earlier quoted context omitted.

I don't know what you're doing with your LLM, but I've only ever had one refusal and I've been working a lot with Claude since it's in bedrock

Comically benign stuff that works fine with GPT-4? It's so trivial to run into Claude lying or responding with arrogant misjudgements. Here's another person's poor anecdotal experiences to pair with yours and mine. [1][2] But more importantly: it shouldn't matter. My tools should not behave this way. Tools should not arbitrarily refuse to work. If I write well-formed C, it compiles , not protests in distaste. If I wr…

Apparently I'm too neurotypical, because I also would agree that judging a person based on only 2 character traits ("Capacity and Intention") is fairly unethical.

I'm sorry neurodiverse people that the world and most humans don't fit into neat categories and systems that you can predict and standardize. And I'm sorry that this makes it harder for you to navigate it. But we get around this problem by recognizing and accommodating the folks that need it, not break the world to fit the desired mold. (i.e. add wheelchair ramps to every building, not force everyone to use a wheelchair)

I realize this is just one example, but it's the one the author chose for that video. (The Cyberpunk thing just seems like a bug.)

To me it seemed like the video was leading up to a 3rd example - of asking Claude about why does japanese culture appreciate precision. THAT would've been a great example - because without any context, that does come off as a racial stereotype (not a negative one, but nonetheless), but for a variety of reasons (covered in the ChatGPT response he included), it IS fairly ubiquitously accurate about Japanese culture, and is worth understanding why. If CLaude had refused to answer this, it would've been a good example of overly arrogant misjudgement.

But he didn't include that, and we can probably guess why - it answered it fine?

I decided to fact check it myself and found out Claude is not yet available in Canada - https://venturebeat.com/ai/anthropic-brings-claude-ai-to-mor...

Re: Claude 2.1

#339

Post by slowmovintarget »

jefftk wrote: post #244
computerfriend wrote: post #241

Earlier quoted context omitted.

Now I know that I can order synthetic virus RNA unscreened. Should your comment be illegal or regulated?

This is a lot like other kinds of security: when there's a hazard out in the wild you sometimes need to make people aware of all or part of the problem as part of fixing it. I would expect making it illegal for people to talk about the holes to make us less safe, since then they never get fixed. This particular hole is not original to me, and is reasonably well known. A group trying to tackle it from a technical pers…

You seemingly dodged the question.

In justifying your post, you actually answered contrary to your original assertion. The information is out there, we should talk about it to get the issue fixed. The same justification applies to avoiding LLM censorship.

There's a sea-change afoot, and having these models in the hands of a very few corporations, aligned to the interests of those corporations and not individuals, is a disaster in the making. Imagine the world in two years... The bulk of the internet will be served up through an AI agent buffer. That'll be the go-to interface. Web pages are soooo last decade.

When that happens, the people controlling the agents control what you see, hear, and say in the digital realm. Who should control the alignment of those models? It's for sure not OpenAI, Microsoft, Google, Meta, or Apple.

Post reply on HN