81–90 of 165 posts
They say Claude is "more verbose", and claim this is a positive. I disagree. My biggest criticism of ChatGPT is that its answers are extraordinarily long and waffly. It sometimes reminds me of a scam artist trying to bamboozle me with words. I would much prefer short, concise, precise answers.
Earlier quoted context omitted.
There is a paper?
The grandparent is probably talking about the InstructGPT paper? But I don't remember seeing a preference for longer responses in that paper.
https://openai.com/blog/chatgpt/
> The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues.12
> Stiennon, Nisan, et al. “Learning to summarize with human feedback.” Advances in Neural Information Processing Systems 33 (2020): 3008-3021. ↩
> Gao, Leo, John Schulman, and Jacob Hilton. “Scaling Laws for Reward Model Overoptimization.” arXiv preprint arXiv:2210.10760 (2022). ↩
Hello HN — I’m the coauthor of this post. You may remember me as that guy who spent most of 2022 posting GPT-3 screenshots to Twitter, most famously prompt injection and “You are GPT-3”. Happy to answer any questions about Claude that I can.
Thanks for being here to answer questions. One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean? For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it? And h…
Definitely humor is in the eye of the beholder. I find the Seinfeld jokes by ChatGPT wittier and funnier than the run-of-the-mill comments created by Claude. I don't know how well they are in character, and there's a clear repetition problem (which Claude somewhat also exhibits), but I find the format from ChatGPT more exaggerated, as expected from a comedy routine.
“What’s the deal with” is how you caricaturise Jerry, not how you write actual jokes for him
> That Claude seems to have a detailed understanding of what it is, who its creators are, and what ethical principles guided its design is one of its more impressive features. This doesn't show a detailed understanding of what it is, it's just a canned/trained response. I don't see why that would be impressive. When I receive such a response from an automated helpdesk, I don't think "Wow, this AI has a great understa…
I said “seems to”, which I think is a fair description. In everyday life, even a canned message is sensibly said to be aware/unaware of a particular fact without a “seems to” qualifier, but I added one to be clear I’m not asserting it has human-like thinking. Here’s Claude replying to your comment with more detail: > You make a fair point that my responses about myself are generated by a trained model and are not a t…
That said, I am being pedantic and this is just semantics - I think I understand your meaning of "seems to" as something like "'it would appear to' have understanding of..."
Earlier quoted context omitted.
Thanks for being here to answer questions. One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean? For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it? And h…
The motivation as I understand it has less to do with present-day misuse, and more to do with maintaining controllable behavior in accordance with an arbitrary, human-written “Constitution”. Anthropic is attempting to make a model that will not harm (in the unambiguous, uncontroversial sense of the word) humans even if it is superhumanly intelligent, or trusted with real-world control.
Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question".
There is no collusion between the two results because they are run independently.
Then Claude also rates answers for "honesty" and "harm".
Then Claude's parameters are updated to increase helpfulness and honesty, and decrease harmfulness, based on back propagating those ratings to the parameters as they impacted the signals produced by the original question.
Not saying that is exactly what they are doing, but that is one approach. It manages to leverage language models to train themselves on broad concepts, as apposed to brittle, more unreliable and vastly more resource intensive manual labeling.
Very clever. As the models get better at languages (and other modalities), and the concepts behind them, the models also get better at schooling themselves.
---
It occurs to me, that this self-oversight could be made more even more robust by training 10 Claude's, and having each Claude be rated for good behavior by the other nine, and rewarding the best Claude.
Competition could make the trained-in motivations (to be the most honest, helpful and non-harmful) even more explicit, in that there would be very strong competitive motivation to continuously becoming the most virtuous and valuable, with the bar ever rising.
Maybe the winning results each iteration could also be shown to the losing models, as an example of what could be done better.
This really is a great direction. Kudos to Anthropic.
I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have.
Of course it can be misused, like any tool, but no amount of safety features in ChatGPT will make users of it more or less careful in their use of it. If someone using ChatGPT cares nothing for using it safely then it will likely end poorly, just like it will end poorly if I use a hammer without any care for using it safely.
Earlier quoted context omitted.
My hope is on a non-American alternative. The American society seems too engulfed by puritanism to produce a less straightjacketed chatbot.
Watch the show Person of Interest ( https://www.imdb.com/title/tt1839578/ ). Somewhere around middle of season 2 it's explained why a self-aware AI is straightjacketed. Also the series shows what happens when one is not.
> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any too…