Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3 model family

anthropic.com

Post reply on HN
Print view View original

411–420 of 723 posts

Re: Claude 3 model family

#411

Post by bugglebeetle »

HarHarVeryFunny wrote: post #401
bugglebeetle wrote: post #323

Earlier quoted context omitted.

“We have everything” is not “some underlying rights to the tech.” I dunno what the angle is on minimizing here, but I’ll take the head of Microsoft at his word vs. more strained explanations about why this isn’t the case.

The AGI exclusion is well known, for example covered here: https://cryptoslate.com/agi-is-excluded-from-ip-licenses-wit... It's also explicitly mentioned in Musk's lawsuit against OpenAI. Much as Musk wants to claim that OpenAI is a subsidiary of Microsoft, even he has to admit that if in fact OpenAI develop AGI then Microsoft won't have any IP rights to it! The context for Nadella's "We have everything" (without of…

OpenAI has not and will likely never develop AGI, so this is akin to saying “Microsoft doesn’t own OpenAI because they have a clause in their contract that’s says they stop owning it when leprechauns exist.” Musk is trying to argue leprechauns exist because he’s mad he got outmaneuvered by Altman, which I imagine will go as well as you’d expect that argument to go in a court of law.

Re: Claude 3 model family

#412

Post by Terretta »

evantbyrne wrote: post #405
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

It seems like it is getting tripped up on grammar. Do these models not deterministically preparse text input into a logical notation?

No* they are text continuations.

Given a string of text, what's the most likely text to come next.

You /could/ rewrite input text to be more logical, but what you'd actually want to do is rewrite input text to be the text most likely to come immediately before a right answer if the right answer were in print.

* Unless you mean inside the model itself. For that, we're still learning what they're doing.

Re: Claude 3 model family

#413

Post by m00nsome »

j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

To solve this problem, we need to understand the relationships between Sally and her siblings.

Given information: - Sally (a girl) has 3 brothers. - Each brother has 2 sisters.

Since Sally is a girl, and she has 3 brothers, it means that each of her brothers considers her as one of their sisters.

Therefore, if each brother has 2 sisters, and Sally is one of those sisters for each brother, then Sally has 1 other sister besides herself.

So, the number of sisters Sally has is 1.

- from Sonnet

Re: Claude 3 model family

#414

Post by vjerancrnjak »

evantbyrne wrote: post #405
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

It seems like it is getting tripped up on grammar. Do these models not deterministically preparse text input into a logical notation?

There's no preprocessing being done. This is pure computation, from the tokens to the outputs.

I was quite amazed that during 2014-2016, what was being done with dependency parsers, part-of-speech taggers, named entity recognizers, with very sophisticated methods (graphical models, regret minimizing policy learners, etc.) became fully obsolete for natural language processing. There was this period of sprinkling some hidden-markov-model/conditional-random-field on top of neural networks but even that disappeared very quickly.

There's no language modeling. Pure gradient descent into language comprehension.

Re: Claude 3 model family

#415

Post by bevekspldnw »

ignoramous wrote: post #232
Workaccount2 wrote: post #125

Earlier quoted context omitted.

I'd almost say anyone not doing that is being foolish. The goal of the service is to answer complex queries correctly, not to have a pure LLM that can do it all. I think some engineers feel that if they are leaning on an old school classically programed tool to assist the LLM, it's somehow cheating or impure.

> I'd almost say anyone not doing that is being foolish The problem is, such tricks are sold as if there's superior built-in multi-modal reasoning and intelligence instead of taped up heuristics, exacerbating the already amped up hype cycle in the vacuum left behind by web3.

I’m the first to agree LLM are not AGI, but I make extensive use of them to solve real world problems. They have intrinsic value.

web3 on the other hand have zero use cases other than Ponzi schemes.

Are LLM living up to all the hype? No.

Are they a hugely significant technology? Yes.

Are they web3 style bullshit? Not at all.

Re: Claude 3 model family

#416

Post by j-b »

m00nsome wrote: post #413
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

To solve this problem, we need to understand the relationships between Sally and her siblings. Given information: - Sally (a girl) has 3 brothers. - Each brother has 2 sisters. Since Sally is a girl, and she has 3 brothers, it means that each of her brothers considers her as one of their sisters. Therefore, if each brother has 2 sisters, and Sally is one of those sisters for each brother, then Sally has 1 other siste…

I tried Sonnet also, to no avail:

To solve this problem, we need to find the number of sisters Sally has.

Given information:

Sally has 3 brothers. Each brother has 2 sisters. Since Sally is a girl, she is not counted as a sister to her brothers.

Step 1: Find the total number of sisters for all 3 brothers. Number of sisters for each brother = 2 Total number of sisters for all 3 brothers = 3 × 2 = 6

Step 2: Since Sally is not counted as a sister to her brothers, the number of sisters Sally has is the total number of sisters for all 3 brothers minus Sally herself. Number of sisters Sally has = Total number of sisters for all 3 brothers - 1 Number of sisters Sally has = 6 - 1 = 5

Therefore, Sally has 5 sisters.

Re: Claude 3 model family

#417

Post by bbor »

j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

lol that’s actually awesome. I think this is a clear case where the fine tuning/prompt wrapping is getting in the way of the underlying model!

  Each of Sally's brothers has 2 sisters. One of these sisters is Sally, but who is the other sister? Since Sally has only brothers and no other sisters are mentioned, we can conclude that the other sister for each brother is also Sally.
It’s clearly taught to do Chain of Reasoning out of the box, but typing it out tricked it because of the short, declarative sentences trying to establish something like “individual” facts. Poor Anthropic!

Re: Claude 3 model family

#418

Post by bbor »

gait2392 wrote: post #409
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

YOU answered it incorrectly. The answer is 1. I guess Claude can comprehend the answer better than (some) humans

They know :). They posted a transcript of their conversation. Claude is the one that said “0”.

Re: Claude 3 model family

#419

Post by nicklevin »

rendang wrote: post #328
mattlondon wrote: post #316

Another naming disaster! Opus is better than sonnet? And sonnet is better than haiku? Perhaps this makes sense to people familiar with sonnets and haikus and opus....es? Nonsensical to me! I know everyone loves to hate on Google, but at least pro and ultra have a sort of sense of level of sophistication.

I think the intention was more "bigger" than better - but opus is an odd choice. haiku>sonnet>ballad maybe? haiku>sonnet>epic?

The EHR company Epic uses a similar naming scheme for the slimmed down version of their EHR (Sonnet) and mobile app (Haiku). Their Apple Watch app is Limerick.

Re: Claude 3 model family

#420

Post by bbor »

evantbyrne wrote: post #405
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

It seems like it is getting tripped up on grammar. Do these models not deterministically preparse text input into a logical notation?

No - that’s the beauty of it. The “computing stack” as taught in Computer Organization courses since time immemorial just got a new layer, imo: prose. The whole utility of these models is that they operate in the same fuzzy, contradictory, perspective-dependent epistemic space that humans do.

Phrasing it like that, it sounds like the stack has become analog -> digital -> analog, in a way…

Post reply on HN