Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Haiku 4.5

anthropic.com

Post reply on HN
Print view View original

281–290 of 292 posts

Re: Claude Haiku 4.5

#281

Post by tempusalaria »

simonw wrote: post #27
tempusalaria wrote: post #26

Earlier quoted context omitted.

I vastly prefer the manual caching. There are several aspects of automatic caching that are suboptimal, with only moderately less developer burden. I don’t use Anthropic much but I wish the others had manual cache options

What's sub-optimal about the OpenAI approach, where you get 90% discount on tokens that you've previously sent within X minutes?

Lots of situations, here are 2 I’ve faced recently (cannot give too much detail for privacy reasons, but should be clear enough)

1) low latency desired, long user prompt 2) function runs many parallel requests, but is not fired with common prefix very often. OpenAI was very inconsistent about properly caching the prefix for use across all requests, but with Anthropic it’s very easy to pre-fire

Re: Claude Haiku 4.5

#282

Post by tacone »

Fergusonb wrote: post #278
tosh wrote: post #228

Earlier quoted context omitted.

One of the main issues I had with Claude Code (maybe it‘s the harness?) was that the agent tends to NOT read enough relevant code before it makes a change. This leads to unnecessary helper functions instead of using existing helper functions and so on. Not sure if it is an issue with the models or with the system prompts and so on or both.

I agree, claude is an impressive agent but it seems like it's impatient and trying to make its own thing, tries to make its own tests when I already have them, etc. Maybe better for a new project. GPT 5 (at least with cline) reads whatever you give it, then laser targets the required changes. With High, as long as I actually provided enough relevant context it usually one shots the solution and sometimes even finds t…

> agree, claude is an impressive agent but it seems like it's impatient and trying to make its own thing, tries to make its own tests when I already have them, etc. Maybe better for a new project.

Nope, Claude will deviate from it's own project as well.

Claude is brilliant but needs hard rules. You have to treat it and make it feel like the robot it really. Feed it a bit too much human prose in your instructions and it will start to behave like a teen.

Re: Claude Haiku 4.5

#283

Post by camel_Snake »

Topfi wrote: post #222
deadbabe wrote: post #47

Earlier quoted context omitted.

Those numbers don’t mean anything without average token usage stats.

Fair point of course and it is still far to early to make a definitive statement, but in my still limited experience throughout the night, I have seen Haiku 4.5 be far better in using what I'd consider a justifiable amount of input tokens over e.g. GPT-5 models. Sonnets recent versions also had been better on this front over OpenAIs current best, but I try (not always succeed) to take prior experience and expectation…

> GLM-4.6 is great value but still not solid enough for tool calls, not that fast, etc. so if you can afford something more reliable I'd go for that, but encouraging.

Funny you should say that, because while it is a large model the GLM 4.5 is at the top of Berkley's Function Calling Leaderboard [0] and has one of the lowest costs. Can't comment on speed compared to those smaller models, but the Air version of 4.5 is similarly highly-ranked.

[0]https://gorilla.cs.berkeley.edu/leaderboard.html

Re: Claude Haiku 4.5

#284

Post by simonw »

thefroh wrote: post #269
simonw wrote: post #208

Earlier quoted context omitted.

I thought OpenAI would still handle case? Their cache would work up to the end of the file and you would then pay for uncached tokens for the user's question. Have I misunderstood how their caching works?

not if call #1 is the file + the question, call #2 is the file + a different question, no. if call #1 is the file, call #2 is the file + the question, call #3 is the file + a different question, then yes. and consider that "the file" can equally be a lengthy chat history, especially after the cache TTL has elapsed.

I vibe-coded up a quick UI for exploring this: https://tools.simonwillison.net/prompt-caching

As far as I can tell it will indeed reuse the cache up to the point, so this works:

Prompt A + B + C - uncached

Prompt A + B + D - uses cache for A + B

Prompt A + E - uses cache for A

Re: Claude Haiku 4.5

#285

Post by edmundsauto »

caymanjim wrote: post #197
edmundsauto wrote: post #191

Earlier quoted context omitted.

I don’t mean this with snark, but with age. It’s actually totally cool to not upgrade and then you have stability in your tooling. I bet there is some hella good art being made with Photoshop 6.0 from the 90s right now. The upgrade path is like the technical hedonistic treadmill. You don’t have to upgrade.

Almost all my tooling is years (or decades) old and stable. But the code assistant LLM scene effectively didn't exist in any meaningful way until this year, and it changes almost daily. There is no stability in the tooling, and you're missing out if you don't switch to newer models at least every few weeks right now. Codex (OpenAI/ChatGPT CLI) didn't even exist a month ago, and it's a contender for the best option. C…

> because you're crippling yourself if you don't

What makes you say this, practically?

Re: Claude Haiku 4.5

#286

Post by lukeck »

squigz wrote: post #214
peddling-brink wrote: post #174

Earlier quoted context omitted.

lol, I don’t know if you work there or not, but directing folks to send their account info to a random Twitter address is, not considered best practice.

What's wrong with sending a username to someone?

Generally, nothing inherently wrong with sending a username but directing people to a 3rd party social media platform rather than an official Anthropic email or support system does nothing to build trust that they actually work there.

Re: Claude Haiku 4.5

#287

Post by 1659447091 »

benjiro wrote: post #239
1659447091 wrote: post #154

Earlier quoted context omitted.

> Ain't nobody got time to pick models and compare features. ... Make it integrate in a generic way, ... , so that it doesn't matter whether I'm using a CLI or neovim or an IDE, and so that I don't have to constantly switch tooling. I use GitHub Copilot Pro+ because this was my main requirement as well. Pro+ has the new models as they come out -- actually just enabled Claude Haiku 4.5 for selection availability. I ha…

> I use GitHub Copilot Pro+ because this was my main requirement as well. Frankly, i do not even get how people run out of 1500 requests. For a heavy coding session, my max is around 45 requests per day, and that means a ton of code / alterations and some wasted on fluff mini changes. Most days is barely in the 10 a 20. I noticed that you can really eat your requests if you just do not care to switch models for small…

> To be honest, i wish that Copilot had a 600 token version, instead of the massive jump to 1500. Other option is to just use the pay per request.

Same here. Well, 900 would be a good middle option for me as well. I was switching to the unlimited model for the simple things, but since I don't use all of the premium allotment I started just leaving it on the one that is working best for the job that day.

I guess part of the "value" of Pro+ is the extra "Spark" credits of which I have zero use for. But I simply wanted something that integrated into my ecosystem instead of having to add to/or change it. Also did not want to have to think about how many pennies I'm using (I appreciate that breakdown though! good to know) -- I'll pay a reasonable convenience tax for my time and mental space of not having to babysit usage.

Re: Claude Haiku 4.5

#289

Post by theshrike79 »

thomasfromcdnjs wrote: post #249
tosh wrote: post #228

Earlier quoted context omitted.

One of the main issues I had with Claude Code (maybe it‘s the harness?) was that the agent tends to NOT read enough relevant code before it makes a change. This leads to unnecessary helper functions instead of using existing helper functions and so on. Not sure if it is an issue with the models or with the system prompts and so on or both.

You might get better results with https://github.com/oraios/serena I sometimes use it, but I've found just adding to my claude.md something like "if you ever refactor code, try search around the codebase to see if their is an existing function you can use or extend"

This and also give it a tool (script or a fd/rg snippet) it can use to find all functions and their documentation based on a criteria

Re: Claude Haiku 4.5

#290

Post by Topfi »

camel_Snake wrote: post #283
Topfi wrote: post #222

Earlier quoted context omitted.

Fair point of course and it is still far to early to make a definitive statement, but in my still limited experience throughout the night, I have seen Haiku 4.5 be far better in using what I'd consider a justifiable amount of input tokens over e.g. GPT-5 models. Sonnets recent versions also had been better on this front over OpenAIs current best, but I try (not always succeed) to take prior experience and expectation…

> GLM-4.6 is great value but still not solid enough for tool calls, not that fast, etc. so if you can afford something more reliable I'd go for that, but encouraging. Funny you should say that, because while it is a large model the GLM 4.5 is at the top of Berkley's Function Calling Leaderboard [0] and has one of the lowest costs. Can't comment on speed compared to those smaller models, but the Air version of 4.5 is…

Gorilla is a great resource and it isn't unreasonable to suspect Z.AI has it in their data sets. I'd suspect most other frontier labs as well (pure speculation, but why not use it as a resource).

Problem is, while Gorilla was an amazing resource back in 2023 and continues to be a great dataset to lean on, but most ways we use LLMs in multi step tasks have since evolved greatly, not just with structured JSON (which GorillaOpenFunctionsV2, v4 eval does multi too), but more with the scaffolding around models (Claude Code vs Codex vs OpenCode, etc.). Likely why good performance with Gorilla doesn't necessarily map onto multiple step workloads with day-to-day tooling, which I tend to go for and reason why, despite there being FOSS options already, most labs either built their own coding assistant tooling (and most open source that too) or feel the need to fork others (Qwen with Geminis repo).

Purely speculative, but GLM-4.6 I evaluated using the same tasks as other models via Claude Code with their endpoint as that is what they advertise as the official way to use the model, same reason I use e.g. Codex for GPT-5. More focused on results in the best case, over e.g. using opencode for all models to give a more level playing field.

Post reply on HN