Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3 model family

anthropic.com

Post reply on HN
Print view View original

121–130 of 723 posts

Re: Claude 3 model family

#121

Post by nopinsight »

hackerlight wrote: post #51
up6w6 wrote: post #22

The Opus model that seems to perform better than GPT4 is unfortunately much more expensive than the OpenAI model. Pricing (input/output per million tokens): GPT4-turbo: $10/$30 Claude 3 Opus: $15/$75

Their smallest model outperforms GPT-4 on Code. I'm sceptical that it'll hold up to real world use though.

Just a note that the 67.0% HumanEval figure for GPT-4 is from its first release in March 2023. The actual performance of current ChatGPT-4 on similar problems might be better due to OpenAI's internal system prompts, possible fine-tuning, and other tricks.

Re: Claude 3 model family

#122

Post by hobofan »

jaysinn_420 wrote: post #104
drpossum wrote: post #96

One of my standard questions is "Write me fizzbuzz in clojure using condp". Opus got it right on the first try. Most models including ChatGPT have flailed at this as I've done evaluations. Amazon Bedrock when?

https://www.aboutamazon.com/news/aws/amazon-bedrock-anthropi... Now...

Or you could go to the primary source (= the article this discussion is about):

> Sonnet is also available today through Amazon Bedrock and in private preview on Google Cloud’s Vertex AI Model Garden—with Opus and Haiku coming soon to both.

Re: Claude 3 model family

#123

Post by simonw »

I'm trying to access this via the API and I'm getting a surprising error message:

Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'max_tokens: 100000 > 4096, which is the maximum allowed value for claude-3-opus-20240229'}}

Maximum tokens of 4096 doesn't seem right to me.

UPDATE: I was wrong, that's the maximum output tokens not input tokens - and it's 4096 for all of the models listed here: https://docs.anthropic.com/claude/docs/models-overview#model...

Re: Claude 3 model family

#124

Post by entrep »

toxik wrote: post #110

Europeans, don't bother signing up - it will not work and it will only tell you once it has your e-mail registered.

If you choose API access you can sign up and verify your EU phone number to get $5 credits

Re: Claude 3 model family

#125

Post by Workaccount2 »

anonymouse008 wrote: post #87
virgildotcodes wrote: post #68

Just signed up for Claude Pro to try out the Opus model. Decided to throw a complex query at it, combining an image with an involved question about SDXL fine tuning and asking it to do some math comparing the cost of using an RTX 6000 Ada vs an H100. It made a lot of mistakes. I provided it with a screenshot of Runpod's pricing for their GPUs, and it misread the pricing on an RTX 6000 ADA as $0.114 instead of $1.14.…

I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue. For reference, Let's build the GPT Tokenizer https://www.youtube.com/watch?v=zduSFxRajkE

I'd almost say anyone not doing that is being foolish.

The goal of the service is to answer complex queries correctly, not to have a pure LLM that can do it all. I think some engineers feel that if they are leaning on an old school classically programed tool to assist the LLM, it's somehow cheating or impure.

Re: Claude 3 model family

#127

Post by dangond »

kylebenzle wrote: post #105
jimbokun wrote: post #93

If you showed someone this article 10 years ago, they would say it indicates Artificial General Intelligence has arrived.

1. It's an advertisement/press release, not so much an "article". 2. This would NOT be called even "AI" but "machine learning" 10 years ago. We started using AI as a marketing term for ML about a year ago.

This absolutely would be called AI 10 years ago. Yes, it's a machine learning task, but a computer program you can speak with would certainly qualify as AI to anyone 10 years ago, if not several decades prior as well.

Re: Claude 3 model family

#128

Post by humanistbot »

maelito wrote: post #117
toxik wrote: post #110

Europeans, don't bother signing up - it will not work and it will only tell you once it has your e-mail registered.

Why is that ? Thanks for the tip that will help 700 million people.

They don't want to comply with the GDPR or other EU laws.

Re: Claude 3 model family

#129

Post by Workaccount2 »

behnamoh wrote: post #119
jimbokun wrote: post #93

If you showed someone this article 10 years ago, they would say it indicates Artificial General Intelligence has arrived.

That's the good thing about intelligence: We have no fucking clue how to define it, so the goalpost just keeps moving.

I'd argue the goalpost is already past what some, albeit small, group of humans are capable of.

Re: Claude 3 model family

#130

Post by brainless »

wesleyyue wrote: post #62

Just added Claude 3 to Chat at https://double.bot if anyone wants to try it for coding. Free for now and will push Claude 3 for autocomplete later this afternoon. From my early tests this seems like the first API alternative to GPT4. Huge!

Hey Wesley, I just checked Double. Do you plan to support open source models hosted locally or on a cloud instance? Asking out of curiosity as I am building a product in the same space and have had a few people ask this. I guess since Double is an extension in IDEs, it can connect to AI models running anywhere.
Post reply on HN