Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3 model family

anthropic.com

Post reply on HN
Print view View original

401–410 of 723 posts

Re: Claude 3 model family

#401

Post by HarHarVeryFunny »

bugglebeetle wrote: post #323
HarHarVeryFunny wrote: post #308

Earlier quoted context omitted.

Sure, but that's just saying that Microsoft as investor has some rights to the underlying tech. There are limits to this though, which we may fairly soon be nearing. I believe the agreement says that Microsoft's rights to the tech (model + weights? training data? -- not sure how specific it is) end once AGI is achieved, however that is evaluated. But again, this is not to say that OpenAI is "Microsoft in a trenchcoat…

“We have everything” is not “some underlying rights to the tech.” I dunno what the angle is on minimizing here, but I’ll take the head of Microsoft at his word vs. more strained explanations about why this isn’t the case.

The AGI exclusion is well known, for example covered here:

https://cryptoslate.com/agi-is-excluded-from-ip-licenses-wit...

It's also explicitly mentioned in Musk's lawsuit against OpenAI. Much as Musk wants to claim that OpenAI is a subsidiary of Microsoft, even he has to admit that if in fact OpenAI develop AGI then Microsoft won't have any IP rights to it!

The context for Nadella's "We have everything" (without of course elaborating on what "everything" referred to) is him trying to calm investors who were just reading headlines about OpenAI imploding in reaction to the board having fired Altman, etc. Nadella wasn't lying - he was just being coy about what "everything" meant, wanting to reassure investors that their $10B investment in OpenAI had not just gone up in smoke.

Re: Claude 3 model family

#404

Post by [deleted] »

SirMaster wrote: post #386
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

mistralai/Mixtral-8x7B-Instruct-v0.1 got this right. >Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Sally has 1 sister. Here's the logic behind this: 1. We know that Sally is a girl and she has 3 brothers. 2. Then we are told that each of her brothers has 2 sisters. 3. Since all of Sally's brothers share the same siblings, they would both count Sally as one of their two…

[deleted]

Re: Claude 3 model family

#405

Post by evantbyrne »

j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

It seems like it is getting tripped up on grammar. Do these models not deterministically preparse text input into a logical notation?

Re: Claude 3 model family

#406

Post by data-ottawa »

anonymouse008 wrote: post #87
virgildotcodes wrote: post #68

Just signed up for Claude Pro to try out the Opus model. Decided to throw a complex query at it, combining an image with an involved question about SDXL fine tuning and asking it to do some math comparing the cost of using an RTX 6000 Ada vs an H100. It made a lot of mistakes. I provided it with a screenshot of Runpod's pricing for their GPUs, and it misread the pricing on an RTX 6000 ADA as $0.114 instead of $1.14.…

I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue. For reference, Let's build the GPT Tokenizer https://www.youtube.com/watch?v=zduSFxRajkE

ChatGPT definitely has a growing bag of tricks like that.

When I use analysis mode to generate and evaluate code it recently started writing the code, then introspecting it and rewriting the code with an obvious hidden step asking "is this code correct". It made a huge improvement in usability.

Fairly recently it would require manual intervention to fix.

Re: Claude 3 model family

#407

Post by bevekspldnw »

anonymouse008 wrote: post #87
virgildotcodes wrote: post #68

Just signed up for Claude Pro to try out the Opus model. Decided to throw a complex query at it, combining an image with an involved question about SDXL fine tuning and asking it to do some math comparing the cost of using an RTX 6000 Ada vs an H100. It made a lot of mistakes. I provided it with a screenshot of Runpod's pricing for their GPUs, and it misread the pricing on an RTX 6000 ADA as $0.114 instead of $1.14.…

I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue. For reference, Let's build the GPT Tokenizer https://www.youtube.com/watch?v=zduSFxRajkE

You can often see it write and execute python code to answer a question which is awesome.

Re: Claude 3 model family

#408

Post by visarga »

HarHarVeryFunny wrote: post #333
visarga wrote: post #309

Earlier quoted context omitted.

If you invented the transformer but didn't have trillions of tokens to train it with, no chatGPT. But if you had Mamba/RWKV/SSSM and trillions of tokens you would have almost the same thing with chatGPT. The training set is magical. It took humanity a long time to discover all the nifty ideas we have in it. It's the result of many generations of humans working together, using language to share their experience. Intel…

I agree about language - which might be though of as "thought macros". Human experience has taught us what things (objects, actions, etc) are worth labelling, what thought patterns are useful to reason about them, etc. Being able to reason about things in the realm of, and using the patterns of, human language is tremendously powerful. Are there any Mamba benchmarks that show it matching transformer (GPT, say) benchm…

I don't think there are Mamba LLMs larger than 2.8B at the moment. But here a crop of papers building on it, mostly vision applications:

https://trendingpapers.com/search?q=mamba

Re: Claude 3 model family

#409

Post by gait2392 »

j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

YOU answered it incorrectly. The answer is 1. I guess Claude can comprehend the answer better than (some) humans

Re: Claude 3 model family

#410

Post by bbor »

phkahler wrote: post #383
j-b wrote: post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

This is why I doubt all the AI hype. These things are supposed to have PhD level smarts, but the above example can't reason about the problem well at all. There's a difference between PhD level information and advanced reasoning , and I'm not sure how many people can tell the difference (I'm no expert). In an adjacent area - autonomous driving - I know that lane following is f**ing easy, but lane identification and o…

LLMs are intuitive computing algorithms, which means they only mimic the subconscious faculties of our brain. You’re referencing the need for careful systematic logical self-aware thinking, which is a great point! You’re absolutely right that LLMs can only loosely approximate it on their own, and not that well.

Luckily, we figured out how to write programs to mimic that part of the brain in the 70s ;)

Post reply on HN