221–230 of 525 posts
I used Gemini instead of my usual Claude for a non-trivial front-end project [1] and it really just hit it out of the park especially after the update last week, no trouble just directly emitting around 95% of the application. Now Claude is back! The pace of releases and competition seems to be heating up more lately, and there is absolutely no switching cost. It's going to be interesting to see if and how the fronti…
This is gonna be game-changing for the next 2-4 weeks before they nerf the model. Then for the next 2-3 months people complaining about the degradation will be labeled “skill issue”. Then a sacrificial Anthropic engineer will “discover” a couple obscure bugs that “in some cases” might have lead to less than optimal performance. Still largely a user skill issue though. Then a couple months later they’ll release Opus 4…
Interestingly, I canceled my Claude subscription. I've paid through the first week of December, so it dries up on the 7th of December. As soon as I had canceled, Claude Code started performing substantially better. I gave it a design spec (a very loose design spec) and it one-shotted it. I'll grant that it was a collection of docker containers and a web API, but still. I've not seen that level of performance from Cla…
Very intriguing, curious if others have seen this.
Earlier quoted context omitted.
it's not about the terminal, but about decoupling yourself from looking at the code. The Claude app lets you interact with a github repo from your phone.
This is not the way these agents are not up to the task of writing production level code at any meaningful scale looking forward to high paying gigs to go in and clean up after people take them too far and the hype cycle fades --- I recommend the opposite, work on custom agents so you have a better understanding of how these things work and fail. Get deep in the code to understand how context and values flow and get…
I think the new one is. I could be the fool and be proven wrong though.
The burying of the lede here is insane. $5/$25 per MTok is a 3x price drop from Opus 4. At that price point, Opus stops being "the model you use for important things" and becomes actually viable for production workloads. Also notable: they're claiming SOTA prompt injection resistance. The industry has largely given up on solving this problem through training alone, so if the numbers in the system card hold up under a…
Earlier quoted context omitted.
Interesting. Tell me more.
https://apps.apple.com/us/app/claude-by-anthropic/id64737536... Has a section for code. You link it to your GitHub, and it will generate code for you when you get on the bus so there's stuff for you to review after you get to the office.
Earlier quoted context omitted.
There are two possible explanations for this behavior: the model nerf is real, or there's a perceptual/psychological shift. However, benchmarks exist. And I haven't seen any empirical evidence that the performance of a given model version grows worse over time on benchmarks (in general.) Therefore, some combination of two things are true: 1. The nerf is psychologial, not actual. 2. The nerf is real but in a way that…
> The nerf is psychologial, not actual Once I tested this, I gave the same task for a model after the release and a couple weeks later. In the first attempt it produced a well-written code that worked beautifully, I started to worry about the jobs of the software engineers. Second attempt was a nightmare, like a butcher acting as a junior developer performing a surgery on a horse. Is this empirical evidence? And this…
Look, I'm not defending the big labs, I think they're terrible in a lot of ways. And I'm actually suspending judgement on whether there is ~some kind of nerf happening.
But the anecdote you're describing is the definition of non-empirical. It is entirely subjective, based entirely on your experience and personal assessment.
Earlier quoted context omitted.
> The nerf is psychologial, not actual Once I tested this, I gave the same task for a model after the release and a couple weeks later. In the first attempt it produced a well-written code that worked beautifully, I started to worry about the jobs of the software engineers. Second attempt was a nightmare, like a butcher acting as a junior developer performing a surgery on a horse. Is this empirical evidence? And this…
No, it's entirely psychological. Users are not reliable model evaluators. It's a lesson the industry will, I'm afraid, have to learn and relearn over and over again.
Why in the world, if I'm paying the loss leader price for "unlimited" usage of these models, would any of these companies literally respect my preference to have unfettered access to the most expensive inference?
Especially when one of the hallmark features of GPT-5 was a fancy router system that decides automatically when to use more/less inference resources, I'm very wary of those `/model` settings.
Earlier quoted context omitted.
No, it's entirely psychological. Users are not reliable model evaluators. It's a lesson the industry will, I'm afraid, have to learn and relearn over and over again.
Giving the same prompt resulting in totally different results is not user evaluation. Nor psychological. You cannot tell the customer you are working for as a developer, that hey, first time it did what you asked, second time it ruined everything, but look, here is the benchmark from Antrophic, according to this there is nothing wrong. The only thing that matters and that can evaluate performance is the end result. B…
I've almost ran out of Claude on the Web credits. If they announce that they're going to support Opus then I'm going to be sad :'(