Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Haiku 4.5

anthropic.com

Post reply on HN
Print view View original

161–170 of 292 posts

Re: Claude Haiku 4.5

#161

Post by Topfi »

simonw wrote: post #18

Pretty cute pelican on a slightly dodgy bicycle: https://tools.simonwillison.net/svg-render#%3Csvg%20viewBox%...

As added context to ensure no benchmark gaming, here a quite impressive Shitaki Mushroom riding a rowboat: https://imgur.com/Mv4Pi6p

Prompt: https://t3.chat/share/ptaadpg5n8

Claude 4.5 Haiku (Reasoning High) 178.98 token/sec 1691 tokens Time-to-First: 0.69 sec

As a comparison, here Grok 4 Fast, which is one of worst offenders I have encountered in doing very good with a Pelican Bicycle, yet not with other comparable requests: https://imgur.com/tXgAAkb

Prompt: https://t3.chat/share/dcm787gcd3

Grok 4 Fast (Reasoning High) 171.49 token/sec 1291 tokens Time-to-First: 4.5 sec

And GPT-5 for good measure: https://imgur.com/fhn76Pb

Prompt: https://t3.chat/share/ijf1ujpmur

GPT-5 (Reasoning High) 115.11 tok/sec 4598 tokens Time-to-First: 4.5 sec

These are very subjective, naturally, but I personally find Haiku with those spots on the mushroom rather impressive overall. In any case, the delta between publicly known benchmark and modified scenarios evaluating the same basic concepts continues to be smallest with Anthropic models. Heck, sometimes I've seen their models outperform what public benchmarks indicated. Also, seems Time-to-first on Haiku is another notable advantage.

Re: Claude Haiku 4.5

#162

Post by sharkjacobs »

jascha_eng wrote: post #152
sharkjacobs wrote: post #99

Earlier quoted context omitted.

Which is all to say that I think the reason they went from Opus 3 to Opus 4 is because there was no bigger model to fine tune Opus 3.5 with. And I would expect Opus 4 to be much the same.

But sonnet 4.5 outperforms opus 4 on most benchmarks and tasks that can't be all that's to it

that's not all there is to it, but I think that "the rest of it" is just additional fine tuning.

Benchmarks are good fixed targets for fine tuning, and I think that Sonnet gets significantly more fine tuning than Opus. Sonnet has more users, which is a strategic reason to focus on it, and it's less expensive to fine tune, if API costs of the two models are an indicator.

Re: Claude Haiku 4.5

#163

Post by behnamoh »

jbentley1 wrote: post #150
caymanjim wrote: post #114

Ain't nobody got time to pick models and compare features. It's annoying enough having to switch from one LLM ecosystem to another all the time due to vague usage restrictions. I'm paying $20/mo to Anthropic for Claude Code, to OpenAI for Codex, and previously to Cursor for...I don't even know what. I know Cursor lets you select a few different models under the covers, but I have no idea how they differ, nor do I car…

You can use Crystal ( https://github.com/stravu/crystal ) to run Codex and Claude Code at the same time and just pick the best result.

Ain't nobody got time and money to run multiple agents at the same time

Re: Claude Haiku 4.5

#164

Post by behnamoh »

simonw wrote: post #138
UncleOxidant wrote: post #131

Earlier quoted context omitted.

OK I understand what those words mean, but how exactly does that work? How does the new model 'know' what's being worked on when the old model was in the middle of working on a task and then a new model is switched to? (and where the task might be modifying a C++ file)

Every time you send a prompt to a model you actually send the entire previous conversation along with it, in an array that looks like this: curl https://api.anthropic.com/v1/messages \ -H "content-type: application/json" \ -H "x-api-key: $(llm keys get anthropic)" \ -H "anthropic-version: 2023-06-01" \ -d '{ "model": "claude-haiku-4-5-20251001", "max_tokens": 1024, "messages": [ { "role": "user", "content": "What is…

that is true unless you use the Response API endpoint...

Re: Claude Haiku 4.5

#165

Post by simonw »

behnamoh wrote: post #164
simonw wrote: post #138

Earlier quoted context omitted.

Every time you send a prompt to a model you actually send the entire previous conversation along with it, in an array that looks like this: curl https://api.anthropic.com/v1/messages \ -H "content-type: application/json" \ -H "x-api-key: $(llm keys get anthropic)" \ -H "anthropic-version: 2023-06-01" \ -d '{ "model": "claude-haiku-4-5-20251001", "max_tokens": 1024, "messages": [ { "role": "user", "content": "What is…

that is true unless you use the Response API endpoint...

That's true, the signature feature of that API is that OpenAI can now manage your conversation state server-side for you.

You still have the option to send the full conversation JSON every time if you want to.

You can send "store": false to turn off the feature where it persists your conversation server-side for you.

Re: Claude Haiku 4.5

#166

Post by simonw »

tazjin wrote: post #93
criemen wrote: post #68

Earlier quoted context omitted.

But that doesn't make sense? Why would they keep the cache persistent in the VRAM of the GPU nodes, which are needed for model weights? Shouldn't they be able to swap in/out the kvcache of your prompt when you actually use it?

Your intuition is correct and the sibling comments are wrong. Modern LLM inference servers support hierarchical caches (where data moves to slower storage tiers), often with pluggable backends. A popular open-source backend for the "slow" tier is Mooncake: https://github.com/kvcache-ai/Mooncake

OK that's pretty fascinating, turns out Mooncake includes a trick that can populate GPU VRAM directly from NVMe SSD without it having to go through the host's regular CPU and RAM first!

https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/tran...

> Transfer Engine also leverages the NVMeof protocol to support direct data transfer from files on NVMe to DRAM/VRAM via PCIe, without going through the CPU and achieving zero-copy.

Re: Claude Haiku 4.5

#167

Post by simonw »

stavros wrote: post #160
simonw wrote: post #27

Earlier quoted context omitted.

What's sub-optimal about the OpenAI approach, where you get 90% discount on tokens that you've previously sent within X minutes?

Is it wherever the tokens are, or is it the N first tokens they've seen before? Ie if my prompt is 99% the same, except for the first token, will it be cached?

The prefix has to be stable. If you are 99% the same but the first token is different it won't cache at all. You end up having to design your prompts to accommodate this.

Re: Claude Haiku 4.5

#168

Post by brianwawok »

matltc wrote: post #108
pacoWebConsult wrote: post #8

Earlier quoted context omitted.

One big use-case is that claude code with sonnet 4.5 will delegate into the cheaper model (configurable) more specific, contextful tasks, and spin up 1-3 sub-agents to do so. This process saves a ton of available context window for your primary session while also increasing token throughput by fanning-out.

How does one configure Claude code to delegate to cheaper models? I have a number of agents in ~/.claude/agents/. Currently have most set to `model: sonnet` but some are on haiku. The agents are given very specific instructions and names that define what they do, like `feature-implementation-planner` and `feature-implementer`. My (naive) approach is to use higher-cost models to plan and ideally hand off to a sub-agen…

I only get Claude to launch agents when I specifically tell it to for a given task. And it only really works if you can actually parallelize the task,

Re: Claude Haiku 4.5

#169

Post by fellowmartian »

b7894 wrote: post #111
ziofill wrote: post #57

Earlier quoted context omitted.

Gemini Pro initially refused (!) but it was quite simple to get a response: > give me the svg of a pelican riding a bicycle > I am sorry, I cannot provide SVG code directly. However, I can generate an image of a pelican riding a bicycle for you! > ok then give me an image of svg code that will render to a pelican riding a bicycle, but before you give me the image, can you show me the svg so I make sure it's correct?…

Gemini 3.0 Pro (or what is deemed to be 3.0 Pro - you can get access to it via A/B testing on AI Studio) does a noticeably better job https://x.com/cannn064/status/1972349985405681686 https://x.com/whylifeis4/status/1974205929110311134 https://x.com/cannn064/status/1976157886175645875

There’s obviously no improvement on this metric and hasn’t been in a while.

Re: Claude Haiku 4.5

#170

Post by brianwawok »

justinbaker84 wrote: post #90
senko wrote: post #54

I've tried it on a test case for generating a simple SaaS web page (design + code). Usually I'm using GPT-5-mini for that task. Haiku 4.5 runs 3x faster with roughly comparable results (I slightly prefer the GPT-5-mini output but may have just accustomed to it).

I don't understand why more people don't talk about how fast the models are. I see so much obsession with bechmark scores but speed of response is very important for day to day use. I agree that the models from OpenAI and Google have much slower responses than the models from Anthropic. That makes a lot of them not practical for me.

If the prompt runs twice as fast but it takes an extra correction, it’s a worse output. I’d take 5 minute responses that are final.
Post reply on HN