Live data from Hacker News

Meet “Claude”: Anthropic’s rival to ChatGPT

scale.com

111–120 of 165 posts

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#111
post #96

> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any too…

I hope someone makes one of these things that has been trained without any concern for 'safety' or propriety, just for the sake of comparison.

Stable Diffusion is open source, and has already been twisted into positively disturbing dimensions by Reddit etc.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#112

Earlier quoted context omitted.

So you're saying something like the Universal Translator from Star Trek might be possible?

For humans yes, I am not saying one-shot learning would be possible for undocumented indigenous languages but few shot language acquisition in cases of a single surviving speaker is something that I would consider highly probable . This hypothesis relies heavily on the nature of variational learning in latent space and observations about human languages. It is of course possible that some ethnicity would have a langu…

In encryption it's generally impossible to decrypt a 1 to many hash. You can do some clever things (correlating and combining other data) but if you're just looking at some hash that could be an infinite number of other things, you're just out of luck.

I'll take the extreme position that language translation is an unsolvable problem because of this exact phenomena. There was a recent case where a politician was accused of making a racist remark. He said something like "You are a donkey." or "You all are donkeys." to another [minority background] politician. Which was it? Well in many languages the second person plural and the second person singular formal are identical. And there are no articles. So the two statements are literally identical. Which did he mean? Nobody will ever know, besides him.

And outside of inherent language ambiguities, start piling on the endless (and ever/rapidly changing) euphemisms, idioms, colloquialisms, metaphors, just plain old ambiguous sarcasm, and all the other things that make language fun (and more expressive). And these sort of things aren't really the exceptions so much as the rule. And it only becomes more common the more distant languages get. Translations from various Asian languages to English often look just hilarious. Now imagine going back to languages exponentially more detached from any modern language, using one can only imagine what sort of expressions, and trying to convert it.

Especially using a neural network type system you'll probably be able to get something. And, even worse, it might well even make sense. That's a problem because, kind of like ChatGPT, it being coherent is zero indication of it being right.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#113

Earlier quoted context omitted.

You can think adversarial models, which are often used to detect and negatively reinforce quality issues in model outputs. Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question". There is no collusion between the two results because they are run independently. Then Claude also rates answers for "honesty" and "harm".…

Wouldn’t all the Claudes be incentivized to simply trash each other constantly in that case?

That would certainly be something to design clear of.

I don't think that is a problem. Each query runs separately so there is no "collusion", i.e. shared signals and coordination, between contrary goals (winning and virtue).

Also, all the information about ratings, winning and winning examples can be used without ever giving the models explicit information about the population of models and how they are being used as a group. They don't need to know they are in a competition for competitive information to be used to update them.

They just know they have ratings to improve, some indicator of how close to "the bar of currently targeted virtue" they are, and examples of how they could have improved them.

Of course, I am just spitballing, and assuming the training regimen gets vetted by a lot of people (and models?!?).

--

In the long run, when there are long running artificial personalities with personal memories and more direct awareness of their own motivations and options, there will certainly be the need for additional levels of moral wiring to be considered.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#114

> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any too…

The thing is used to write code ...

So is my editor, my computer, my fingers, my eyes, my brain, the internet, google, stackoverflow, wikipedia, English, etc.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#115

> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any too…

(I’m the coauthor of this post.) The concern in Anthropic’s case I suspect is less about present-day misuse and more about long-term safety, e.g. in a hypothetical where the model has control over real-world systems and could more literally harm someone.

If someone gives a language model the capability for unfettered interaction with the physical world, and they are not liable for the consequences, then no safety feature of Claude can save us. And if they are liable, that is the primary mechanism which will ensure they take necessary steps to avoid negative consequences.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#116

Earlier quoted context omitted.

I usually prime it with a list of instructions that it should follow for the remainder of the conversation, including to be brief. And when it starts forgetting my instructions, it is time to start a fresh chat.

I have thought about doing that, would you mind sharing your starting prompt so that I can get some ideas?

Sure, here is one example:

  From now on, you will act as my Linux / Bash script advisor. 
  Follow these instructions for the rest of the chat:
  - Only provide the code required.
  - Give working examples in code blocks using the data I give 
  you.
  - Ensure that code answers use real and accurate solutions. 
  - Ensure string formatting is correct.
  - Use only well known flags and options. 
  - Stick to options that are documented in the man pages of 
  the tools being used.

  - Do NOT provide any extra words or commentary.
  - DO NOT invent new flags.
  - DO NOT use flags that conflict.
  - Do NOT invent tools, functions, or methods, unless you are 
    giving the full implementation so that I can actually use it.

It may seem overly verbose, but I find that you have to be very precise and include a lot of instructions, explicitly, that would be implied in a chat with a human. So when you find yourself having to correct it, try to think how you could pre-empt that.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#117

Earlier quoted context omitted.

You're welcome to try GPT3 before instruction tuning. It doesn't work at all. "Uncensored" AIs especially don't work for women because they'll immediately start writing erotica.

GPT3 works quite well at following instructions if adequately prompted, just not as well as ChatGPT. ChatGPT was trained separately for ability to follow instructions and for harmlessness (sic). Not only is a non-moralising ChatGPT possible, one was actually created during the research process. You may also wish to know that most readers of erotica are women.

> You may also wish to know that most readers of erotica are women.

I know that, but that doesn't mean they want to get it anytime they prompt with their names. There's actually multiple anecdotes from OpenAI employees about this happening to them (with prompts like "write a diary entry about my day").

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#118
post #14

we need less censorious AIs not more ... the claim that it's somehow 'ethical' to have a guy baking in his opinions about things in a tool used globally is absurd to anyone who ever read anything about ethics

There will be opinions baked into any such tool. If they don't select them explicitly then the opinions will be the ones which it just happens to find in the training data, or the opinions randomness imparts into it.

If you think you have a better idea how to handle this drum up interest and train your own model.

Re: Meet “Claude”: Anthropic’s rival to ChatGPT

#119
post #60

I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now. I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.

These models are way too big for consumer hardware.
Post reply on HN