Claude 3.5 Sonnet • Hacker News Board

Re: Claude 3.5 Sonnet

#11

Post by mamoul » Thu, Jun 20, 2024, 2:33 PM UTC

swalsh wrote: post #3

Anthropic has been killing it. I subscribe to both chatgpt pro and claude, but I spend probably 90% of my time using Claude. I usually only go back to open ai when I want another model to evaluate or modify the results.

a Kagi Ultimate subscription gets you access to both (plus others) for $25/mo

Re: Claude 3.5 Sonnet

#12

Post by throwup238 » Thu, Jun 20, 2024, 2:36 PM UTC

Woah, this is a marked improvement. I just threw a relative complex coding problem at it and 3.5 sonnet did a really good job across several language. I asked it to rewrite a Qt6 QSyntaxHighlighter subclass to use TreeSitter to support arbitrary languages and not only did it work (with a hardcoded language) but it even got the cxx-qt Rust bindings almost right, including the extra header.

Curious to see how well it handles QML because previous models have been absolutely garbage at it.

Re: Claude 3.5 Sonnet

#13

Post by pantsforbirds » Thu, Jun 20, 2024, 2:37 PM UTC

brokensegue wrote: post #10

Can anyone find pricing details? Ah it is "The model costs $3 per million input tokens and $15 per million output tokens"

Pricing is actually insane for those benchmark results.

Re: Claude 3.5 Sonnet

#14

Post by eigenvalue » Thu, Jun 20, 2024, 2:41 PM UTC

prasoonds wrote: post #8

This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin. Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!

I find that it varies between language and task whether GPT-4o or Claude3 Opus will be better. I usually try both now.

Re: Claude 3.5 Sonnet

#15

Post by 3l3c7r1c » Thu, Jun 20, 2024, 2:41 PM UTC

prasoonds wrote: post #8

This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin. Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!

For coding Claude Opus-3 provides far more mature code and good at finding bugs (when present with the error code) compared to GPT-4-Turbo and GPT-4o. Last few days I've been using both for some python+pyspark project. Not sure how come in their comparison GPT-4o is showing that good!

Re: Claude 3.5 Sonnet

#16

Post by qmarkgn » Thu, Jun 20, 2024, 2:43 PM UTC

The trial requires a signup with an email address. No thanks! This is one thing Microsoft got right with CoPilot.

With so much competition, I wonder why everyone else makes it hard to try out something.

Re: Claude 3.5 Sonnet

#17

Post by eigenvalue » Thu, Jun 20, 2024, 2:43 PM UTC

Awesome, can’t wait to try this. I wish the big AI labs would make more frequent model improvements, like on a monthly cadence, as they continue to train and improve stuff. Also seems like a good way to do A/B testing to see which models people prefer in practice.

Re: Claude 3.5 Sonnet

#18

Post by impulser_ » Thu, Jun 20, 2024, 2:44 PM UTC

Anthropic is the new king. This isn't even Claude 3.5 Opus and it's already super impressive. The speed is insane.

I asked it "Write an in depth tutorial on async programming in Go" and it filled out 8 sections of a tutorial with multiple examples per section before GPT4o got to the second section and GPT4o couldn't even finish the tutorial before quitting.

I been a fan of Anthropic models since Claude 3. Despite the benchmarks people always post with GPT4 being the leader, I always found way better results with Claude 3 than GPT4 especially with responses and larger context. GPT responses always feel computer generated, while Claude 3 felt more humanlike.

Re: Claude 3.5 Sonnet

#19

Post by bcherny » Thu, Jun 20, 2024, 2:45 PM UTC

Awesome!!

Seems like the API docs haven't been updated yet: https://docs.anthropic.com/en/docs/about-claude/models

Re: Claude 3.5 Sonnet

#20

Post by nsingh2 » Thu, Jun 20, 2024, 2:49 PM UTC

prasoonds wrote: post #8

This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin. Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!

GPT4(o) is quite good at advanced math, it's been helpful when I was learning differential geometry. Not sure how Claude compares though, this 3.5 release has tempted me to try it out. Also, it's finally available in Canada!