Claude Sonnet 5 – benchmark results • Hacker News Board

Re: Claude Sonnet 5 – benchmark results

#11

Post by butterisgood » Tue, Jun 30, 2026, 9:20 PM UTC

atemerev wrote: post #5

Yet another mediocre model. Mostly irrelevant among open weights alternatives. Fable wen.

I used sonnet five today to evaluate work I’m doing on an experimental programming language with an interesting concurrency model.

I asked it to try to figure out why one of the examples wasn’t working.

It read the implementation of the compiler and the runtime, found the bug, fixed it, fixed the example and the only thing I had to do manually is suggest a less silly name for a particular function.

I would use sonnet 5 for coding … seems alright!

Re: Claude Sonnet 5 – benchmark results

#12

Post by lucamark » Tue, Jun 30, 2026, 9:23 PM UTC

atemerev wrote: post #5

Yet another mediocre model. Mostly irrelevant among open weights alternatives. Fable wen.

Agree. It is a mediocre model, expensive while not being a frontier

Re: Claude Sonnet 5 – benchmark results

#13

Post by cjk » Tue, Jun 30, 2026, 9:23 PM UTC

CSMastermind wrote: post #7

Using Fable, pretty much every request hit some gate they had for no discernible reason. These provider-level rejections should be incorporated into benchmarks as 0s on the tasks since that's the experience you'll actually get using the model.

I have heard this from a bunch of folks, but that was not my experience. For the couple days I was able to use it, I didn't hit a single gate, and I was using it pretty extensively (but not for anything security-related).

Re: Claude Sonnet 5 – benchmark results

#14

Post by lucamark » Tue, Jun 30, 2026, 9:26 PM UTC

cjk wrote: post #13

CSMastermind wrote: post #7

Using Fable, pretty much every request hit some gate they had for no discernible reason. These provider-level rejections should be incorporated into benchmarks as 0s on the tasks since that's the experience you'll actually get using the model.

I have heard this from a bunch of folks, but that was not my experience. For the couple days I was able to use it, I didn't hit a single gate, and I was using it pretty extensively (but not for anything security-related).

Never had rejections in the short time Fable was available

Re: Claude Sonnet 5 – benchmark results

#15

Post by lucamark » Tue, Jun 30, 2026, 9:27 PM UTC

system2 wrote: post #8

datakan wrote: post #6

I'm so sick of Anthropics usage caps and how their model devours tokens.

It starts with NVIDIA artificially and slowly releasing its tech. If the GPUs were cheaper, we would have better models by many other companies, and competition would take care of these greedy tactics.

Remember that such models are available to us thanks to NVIDIA GPUs

Re: Claude Sonnet 5 – benchmark results

#16

Post by [deleted] » Tue, Jun 30, 2026, 9:38 PM UTC

lucamark wrote: post #15

system2 wrote: post #8

Earlier quoted context omitted.

It starts with NVIDIA artificially and slowly releasing its tech. If the GPUs were cheaper, we would have better models by many other companies, and competition would take care of these greedy tactics.

Remember that such models are available to us thanks to NVIDIA GPUs

[deleted]

Re: Claude Sonnet 5 – benchmark results

#17

Post by [deleted] » Tue, Jun 30, 2026, 9:58 PM UTC

cjk wrote: post #13

CSMastermind wrote: post #7

Using Fable, pretty much every request hit some gate they had for no discernible reason. These provider-level rejections should be incorporated into benchmarks as 0s on the tasks since that's the experience you'll actually get using the model.

I have heard this from a bunch of folks, but that was not my experience. For the couple days I was able to use it, I didn't hit a single gate, and I was using it pretty extensively (but not for anything security-related).

[deleted]

Re: Claude Sonnet 5 – benchmark results

#18

Post by UltraSane » Tue, Jun 30, 2026, 9:59 PM UTC

lucamark wrote: post #14

cjk wrote: post #13

Earlier quoted context omitted.

I have heard this from a bunch of folks, but that was not my experience. For the couple days I was able to use it, I didn't hit a single gate, and I was using it pretty extensively (but not for anything security-related).

Never had rejections in the short time Fable was available

It used Opus for every biology related question I asked it.

Re: Claude Sonnet 5 – benchmark results

#19

Post by olejorgenb » Tue, Jun 30, 2026, 10:28 PM UTC

UltraSane wrote: post #18

lucamark wrote: post #14

Earlier quoted context omitted.

Never had rejections in the short time Fable was available

It used Opus for every biology related question I asked it.

Even opus refuse to discuss micro biology for more than a around 15 turns in my experience.

Re: Claude Sonnet 5 – benchmark results

#20

Post by system2 » Tue, Jun 30, 2026, 11:15 PM UTC

lucamark wrote: post #15

system2 wrote: post #8

Earlier quoted context omitted.

It starts with NVIDIA artificially and slowly releasing its tech. If the GPUs were cheaper, we would have better models by many other companies, and competition would take care of these greedy tactics.

Remember that such models are available to us thanks to NVIDIA GPUs

Thank you NVIDIA for artificially making everything scarce so we pay 5x for everything today.