[dead]
Looking the pricing page, Sonnet 4.6 seems to be about 60% the price of Opus 4.6. What am I missing?
Claude Sonnet 4.6
551–560 of 1001 posts
Re: Claude Sonnet 4.6
#552> Sonnet 4.5, starting at $3/$15 per million tokens. Are people really willing to pay these prices? The open-weight models are catching up in a rapid pace while keeping the prices so low. MiniMax M2.5, Kimi 2.5 and GLM-5 is dirt cheap compared to this. They may not be sota but they are more than good enough.
I made my own benchmarks, very basic questions, and Claude 4.6 is actually worse than the free Stepfun 3.5 version: https://aibenchy.com
It is smart, but it fails at basic instruction following sometimes.
I remember this is a Claude thing for quite a while, where I kept trying to make it output just JSON (without structured output), and it always kept adding quotes or new lines.
Re: Claude Sonnet 4.6
#553I ran the same test I ran on Opus 4.6: feeding it my whole personal collection of ~900 poems which spans ~16 years It is a far cry from Opus 4.6. Opus 4.6 was (is!) a giant leap, the largest since Gemini 2.5 pro. Didn't hallucinate anything and produced honestly mind-blowing analyses of the collection as a whole. It was a clear leap forward. Sonnet 4.6 feels like an evolution of whatever the previous models were doin…
Opus 4.6 is outstanding for code, and for the little I have used it outside of that context, in everything else I have used it with. The productivity with code is at least 3x what I was getting with 5.2, and it can handle entire projects fairly responsibly. It doesn’t patronize the user, and it makes a very strong effort to capture and follow intentions. Unlike 5.2, I’ve never had to throw out a days work that it cov…
That last part is a real one though, mine tried to debug a Dockerfile by poking around my local environment outside of Docker today.
Re: Claude Sonnet 4.6
#554Earlier quoted context omitted.
Isn't "computer use" just interaction with a shell-like environment, which is routine for current agents?
This is being downvoted but it shouldn't be. If the ultimate goal is having a LLM control a computer, round-tripping through a UX designed for bipedal bags of meat with weird jelly-filled optical sensors is wildly inefficient. Just stay in the computer! You're already there! Vision-driven computer use is a dead end.
you could say that about natural language as well, but it seems like having computers learn to interface with natural language at scale is easier than teaching humans to interface using computer languages at scale. Even most qualified people who work as software programmers produce such buggy piles of garbage we need entire software methodologies and testing frameworks to deal with how bad it is. It won't surprise me if visual computer use follows a similar pattern. we are so bad at describing what we want the computer to do that it's easier if it just looks at the screen and figures it out.
Re: Claude Sonnet 4.6
#555Earlier quoted context omitted.
This is being downvoted but it shouldn't be. If the ultimate goal is having a LLM control a computer, round-tripping through a UX designed for bipedal bags of meat with weird jelly-filled optical sensors is wildly inefficient. Just stay in the computer! You're already there! Vision-driven computer use is a dead end.
i replied as much to a sibling comment but i think this is a way to wiggle out of robots.txt, identifying user agent strings, and other traditional ways for sites to filter for a bot.
I mean if they want to "wriggle out" of robots.txt they can just ignore it. It's entirely voluntary.
Re: Claude Sonnet 4.6
#556Earlier quoted context omitted.
Well it is a trick question due to it being non-sensical. The AI is interpreting it in the only way that makes sense, the car is already at the car wash, should you take a 2nd car to the car wash 50 meters away or walk. It should just respond "this question doesn't make any sense, can you rephrase it or add additional information"
How is the question nonsensical? It's a perfectly valid question.
Because validity doesn't depend on meaning. Take the classic example: "What is north of the North Pole?". This is a valid phrasing of a question, but is meaningless without extra context about spherical geometry. The trick question in reference is similar in that its intended meaning is contained entirely in the LLM output.
Re: Claude Sonnet 4.6
#557Earlier quoted context omitted.
Their goal is to monopolize labor for anything that has to do with i/o on a computer, which is way more than SWE. Its simple, this technology literally cannot create new jobs it simply can cause one engineer (or any worker whos job has to do with computer i/o) to do the work of 3, therefore allowing you to replace workers (and overwork the ones you keep). Companies don't need "more work" half the "features"/"products…
I have never been in an organization where everyone was sitting around, wondering what to do next. If the economy was actually as good as certain government officials claimed to be, we would be hiring people left and right to be able to do three times as much work, not firing.
That's the thing, profits and equities are at all time highs, but these companies have laid off 400k SWEs in the last 16 months in the US, which should tell you what their plans are for this technology and augmenting their businesses.
Re: Claude Sonnet 4.6
#558I see a big focus on computer use - you can tell they think there is a lot of value there and in truth it may be as big as coding if they convincingly pull it off. However I am still mystified by the safety aspect. They say the model has greatly improved resistance. But their own safety evaluation says 8% of the time their automated adversarial system was able to one-shot a successful injection takeover even with saf…
This is the elephant in the room nobody wants to talk about. AI is dead in the water for the supposed mass labor replacement that will happen unless this is fixed.
Summarize some text while I supervise the AI = fine and a useful productivity improvement, but doesn’t replace my job.
Replace me with an AI to make autonomous decisions outside in the wild and liability-ridden chaos ensues. No company in their right mind would do this.
The AI companies are now in a extinctential race to address that glaring issue before they run out of cash, with no clear way to solve the problem.
It’s increasingly looking like the current AI wave will disrupt traditional search and join the spell-checker as a very useful tool for day to day work… but the promised mass labor replacement won’t materialize. Most large companies are already starting to call BS on the AI replacing humans en-mass storyline.
Re: Claude Sonnet 4.6
#559Still fails the car wash question, I took the prompt from the title of this thread: https://news.ycombinator.com/item?id=47031580 The answer was "Walk! It would be a bit counterproductive to drive a dirty car 50 meters just to get it washed — you'd barely move before arriving. Walking takes less than a minute, and you can simply drive it through the wash and walk back home afterward." I've tried several other variant…
Looking at the responses below it's interesting how binary they are. It's classic hallucinations style where it's flopping between two alternatives but which ever one it picks it's absolutely confident about.
Re: Claude Sonnet 4.6
#560Earlier quoted context omitted.
Can those be even called open source if you can't rebuild if from the source yourself?
Open weights fulfill a lot of functional the properties of open source, even if not all of them. Consider the classic CIA triad - confidentiality, integrity, and availability. You can achieve all of these to a much greater degree with locally-run open weight models than you can with cloud inference providers. We may not have the full logic introspection capabilities, the ease of modification (though you can still do…
Fair enough but I still prefer people would be more concrete and really call it "open weight" or similar.
With fully open source software (say under GPL3), you can theoretically change anything & you are also quite sure about the provenience of the thing.
With an open weights model you can run it, that is good - but the amount of stuff you can change is limited. It is also a big black box that could possibly hide some surprises from who ever created it that could be possibly triggered later by input.
And lastly, you don't really know what the open weight model was trained on, which can again reflect on its output, not to mention potential liabilities later on if the authors were really care free about their training set.