Still no reduction in price for models capable of Agentic coding over the past year of releases. I'd take the capabilities of the old Sonnet 3.5v2 model if it was ¼ the price of current Sonnet for most situations. But instead of releasing smaller models that are not as smart but still capable when it comes to Agentic coding the price stays the same for the updated minimum viable model.
Claude 4
621–630 of 1001 posts
Re: Claude 4
#622I can't be the only one who thinks this version is no better than the previous one, and that LLMs have basically reached a plateau, and all the new releases "feature" are more or less just gimmicks.
I think they are just getting better at the edges, MCP/Tool Calls, structured output. This definitely isn't increased intelligence, but it an increase in the value add, not sure the value added equates to training costs or company valuations though. In all reality, I have zero clue how any of these companies remain sustainable. I've tried to host some inference on cloud GPUs and its seems like it would be extremely c…
> how any of these companies remain sustainable
They don't, they have a big bag of money they are burning through, and working to raise more. Anthropic is in a better position cause they don't have the majority of the public using their free-tier. But, AFAICT, none of the big players are profitable, some might get there, but likely through verticals rather than just model access.
Re: Claude 4
#623Earlier quoted context omitted.
Context window is unchanged for Sonnet. (200k in/64k out): https://docs.anthropic.com/en/docs/about-claude/models/overv... In practice, the 1M context of Gemini 2.5 isn't that much of a differentiator because larger context has diminishing returns on adherence to later tokens.
I'm going to have to heavily disagree. Gemini 2.5 Pro has super impressive performance on large context problems. I routinely drive it up to 4-500k tokens in my coding agent. It's the only model where that much context produces even remotely useful results. I think it also crushes most of the benchmarks for long context performance. I believe on MRCR (multi round coreference resolution) it beats pretty much any other…
Is that a codebase you're giving it?
Re: Claude 4
#624Earlier quoted context omitted.
When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…
What separates this from humans? Is it unthinkable that LLMs could come up with some response that is genuinely creative? What would genuinely creative even mean? Are humans not also mixing a bag of experiences and coming up with a response? What's different?
Humans brains are animal brains and their primary function is to keep their owner alive, healthy and pass their genes. For that they developed abilities to recognize danger and react to it, among many other things. Language came later.
For a LLM, language is their whole world, they have no body to care for, just stories about people with bodies to care for. For them, as opposed to us, language is first class and the rest is second class.
There is also a difference in scale. LLMs have been fed the entirety of human knowledge, essentially. Their "database" is so big for the limited task of text generation that there is not much left for creativity. We, on the other hand are much more limited in knowledge, so more "unknowns" so more creativity needed.
Re: Claude 4
#625Earlier quoted context omitted.
If you ask an LLM to "act" like someone, and then give it context to the scenario, isn't it expected that it would be able to ascertain what someone in that position would "act" like and respond as such? I'm not sure this is as strange as this comment implies. If you ask an LLM to act like Joffrey from Game of Thrones it will act like a little shithead right? That doesn't mean it has any intent behind the generated o…
Intent at this stage of AI intelligence almost feels beside the point. If it’s in the training data these models can fall into harmful patterns. As we hook these models into more and more capabilities in the real world, this could cause real world harms. Not because the models have the intent to do so necessarily! But because it has a pile of AI training data from Sci-fi books of AIs going wild and causing harm.
Sci-fi books merely explore the possibilities of the domain. Seems like LLMs are able to inhabit these problematic paths, And I'm pretty sure that even if you censor all sci-fi books, they will fall into the same problems by imitating humans, because they are language models, and their language is human and mirrors human psychology.
When an LLM needs to achieve a goal, it invokes goal oriented thinkers and texts, including Machiavelli for example. And its already capable of coming up with various options based on different data.
Sci-fi books give it specific scenarios that play to its strengths and unique qualities, but without them it will just have to discover these paths on its own pace, the same way sci-fi writers discovered them.
Re: Claude 4
#626This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…
When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…
> LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text.
LLMs have a million plans and a million thoughts: they need to simulate all the characters in their text to complete these texts, and those characters (often enough) behave as if they have plans and thoughts.
Compare https://gwern.net/fiction/clippy
Re: Claude 4
#627Earlier quoted context omitted.
When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…
"They do not have a plan" Not necessarily correct if you consider agent architectures where one LLM would come up with a plan and another LLM executes the provided plan. This is already existing.
Yes, it's still correct. Using the wrong words for things doesn't make them magical machine gods.
Re: Claude 4
#628Earlier quoted context omitted.
When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…
What separates this from humans? Is it unthinkable that LLMs could come up with some response that is genuinely creative? What would genuinely creative even mean? Are humans not also mixing a bag of experiences and coming up with a response? What's different?
> What separates this from human.
A lot. Like an incredible amount. A description of a thing is not the thing.
There is sensory input, qualia, pleasure & pain.
There is taste and judgement, disliking a character, being moved to tears by music.
There are personal relationships, being a part of a community, bonding through shared experience.
There is curiosity and openeness.
There is being thrown into the world, your attitude towards life.
Looking at your thoughts and realizing you were wrong.
Smelling a smell that resurfaces a memory you forgot you had.
I would say the language completion part is only a small part of being human.
Re: Claude 4
#629This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…
It's not wild, it's literally every bit of fiction about "how would an AI keep itself alive" of course it's going to settle into that probabilistic path. It's also nonsensical if you think for even one second about the way the program actually runs though.
Why is it nonsense exactly?
Re: Claude 4
#630> Users requiring raw chains of thought for advanced prompt engineering can contact sales So it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn't. In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opi…
Because it's alchemy and everyone believes they have an edge on turning lead into gold.