431–440 of 723 posts
Um, this is starting to become a trend, OpenAI.
Earlier quoted context omitted.
The guard rails on the models make the llm-market a complete train wreck. Wish we could just collectively grow up and accept that if a computer says something bad that doesn't have any negative real world impact - unless we let it - just like literally any other tool.
They're not there to protect the user, they're they're to protect the brand of the provider. A bot that spits out evil shit easily screenshotted with the company's brand right there, isn't really great for growth or the company's brand both.
It's frustrating though because these companies have the resources to do amazing things, but it's been shown that censoring an LLM can dumb it down in general, beyond what it was originally censored for.
Also, this of course. It's just a cheap bandaid to prevent the most egregious mistakes and embarrasing screenshots.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
This is why I doubt all the AI hype. These things are supposed to have PhD level smarts, but the above example can't reason about the problem well at all. There's a difference between PhD level information and advanced reasoning , and I'm not sure how many people can tell the difference (I'm no expert). In an adjacent area - autonomous driving - I know that lane following is f**ing easy, but lane identification and o…
That’s why a lot (though not all!) of the over-the-top LLM hype you see online is coming from people with very little experience and no serious expertise in a technical domain.
If it walks like a duck, and quacks like a duck…
…possibly it’s just an LLM trained on the output of real ducks, and you’re not a duck so you can’t tell the difference.
I think LLMs are simply a less general technology than we (myself included) might have predicted at first interaction. They’re incredibly good at what they do — fluidly manipulating and interpreting natural language. But humans are prone to believing that anything that can speak their language to a high degree of fluency (in the case of GPT-3+, beyond almost all native speakers) must also be hugely intelligent and therefore capable of general reasoning. And in LLMs, we finally have the perfect counterexample.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
Did you perhaps run this on Sonnet?
Opus just crushed Gemini Pro and GPT4 on a pretty complex question I have asked all of them, including Claude 2. It involved taking a 43 page life insurance investment pdf and identifying various figures in it. No other model has gotten close. Except for Claude 3 sonnet, which just missed one question.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
Okay, let's think through this step-by-step:
We know that Sally is a girl and she has 3 brothers. Each of Sally's brothers has 2 sisters. Now, who are these sisters? They must be Sally and one other sister, because if Sally's brothers had any other sisters, they would also be Sally's sisters, and the problem doesn't mention that. So, if each of Sally's brothers has 2 sisters, and one of these sisters is Sally herself, then Sally must have 1 other sister. Therefore, Sally has 1 sister.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
That is not to say this is fine, but more that we tend to get hung up on what these models do wrong rather than all the amazing stuff they do correctly.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
Sally has 1 sister. This may seem counterintuitive at first, but let's reason through it:
We know that Sally has 3 brothers, and she is one of the sisters.
Then we are told that each brother has 2 sisters.
Since Sally's brothers share the same parents as Sally, they share the same sisters.
Therefore, Sally's 3 brothers have only 1 additional sister besides Sally, making Sally's sister count 1.
Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
To solve this problem, we need to understand the relationships between Sally and her siblings. Given information: - Sally (a girl) has 3 brothers. - Each brother has 2 sisters. Since Sally is a girl, and she has 3 brothers, it means that each of her brothers considers her as one of their sisters. Therefore, if each brother has 2 sisters, and Sally is one of those sisters for each brother, then Sally has 1 other siste…