Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4

anthropic.com

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: Claude 4

#911

Post by speed_spread »

mellow_observer wrote: post #560
SamBam wrote: post #545

This is the first LLM that has been able to answer my logic puzzle on the first try without several minutes of extended reasoning. > A man wants to cross a river, and he has a cabbage, a goat, a wolf and a lion. If he leaves the goat alone with the cabbage, the goat will eat it. If he leaves the wolf with the goat, the wolf will eat it. And if he leaves the lion with either the wolf or the goat, the lion will eat the…

Actual answer: He crosses the river and takes all of the animals and the cabbage with him in one go. why not? Alternative Answer: He just crosses the river. Why would he care who eats what? Another Alternative Answer: He actually can't cross the river since he doesn't have a boat and neither the cabbage nor the animals serve as appropriate floatation aids

I will trust LLMs the day they suggest making a raft out of the lion's skin and propel it across the river by eating the cabbage and farting away. Goats are cool, keep the goat.

Re: Claude 4

#912

Post by neal_ »

rxtexit wrote: post #900
hsn915 wrote: post #561

Earlier quoted context omitted.

Hence, "plateau"

"I don't have anything to ask the model, so the model hasn't improved" Brilliant! I am pretty much ready to be done talking to human idiots on the internet. It is just so boring after talking to these models.

:p

Re: Claude 4

#913

Post by rxtexit »

brulard wrote: post #425
tristanb wrote: post #346

Earlier quoted context omitted.

Nice - it might know about Svelte 5 finally...

It knows about Svelte 5 for some time, but it particularly likes to mix it with Svelte 4 in very weird and broken ways.

I have experienced this for various libraries. I think it helps to paste in a package.json in the prompt.

All the models seem to struggle with React three fiber like this. Mixing and matching versions that don't make sense. I can see this being a tough problem given the nature of these models and the training data.

I am going to also try to start giving it a better skeleton to start with and stick to the particular imports when faced with this issue.

My very first prompt with claude 4 was for R3F and it imported a depreciated component as usual.

We can't expect the model to read our minds.

Re: Claude 4

#914

Post by hansmayer »

rxtexit wrote: post #908
lechatonnoir wrote: post #434

Earlier quoted context omitted.

This is a weirdly cherry-picked example. The gaming approach was also the premise of DeepMind's AI efforts in 2016, which was nine years ago. Regardless of what you think about the utility of text (code), video, audio, and image generation, surely you think that their progress on the protein-folding problem and weather prediction have been useful to society? What counts as a killer app to you? Can you name one?

The whole idea of a "killer app" is stupid. It is a dismissive rhetorical device to prove a wrong point on an internet forum such as this that has nothing to do with reality.

Are you sure you are not describing your own argument here?

Re: Claude 4

#915

Post by andrepd »

a_bonobo wrote: post #803
Doohickey-d wrote: post #48

> Users requiring raw chains of thought for advanced prompt engineering can contact sales So it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn't. In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opi…

Could the exclusion of CoT that be because of this recent Anthropic paper? https://assets.anthropic.com/m/71876fabef0f0ed4/original/rea... >We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in the prompts and find: (1) for most settings and models tested, CoTs reveal their usage of hints in at least 1% of examples where they use the hint, but the reveal rate is often…

I have no idea what this means, can someone give the eli5?

Re: Claude 4

#916

Post by jinushaun »

pjmlp wrote: post #841
bredren wrote: post #270

Earlier quoted context omitted.

It still matters for software packages. Particularly python packages that have to do with programming with AI! They are evolving quickly, with deprecation and updated documentation. Having to correct for this in system prompts is a pain. It would be great if the models were updating portions of their content more recently than others. For the tailwind example in parent-sibling comment, should absolutely be as up to d…

Given that I am still coding against Java 17, C# 7, C++17 and such at most work projects, and more recent versions are still the exception, it is quite reasonable. Few are on jobs where v-latest is always an option.

It’s not about the language. I get bit when they recommend old libraries or hallucinate non-existent ones.

Re: Claude 4

#917

Post by riwsky »

XCSme wrote: post #577
_peregrine_ wrote: post #396

Already test Opus 4 and Sonnet 4 in our SQL Generation Benchmark ( https://llm-benchmark.tinybird.live/ ) Opus 4 beat all other models. It's good.

It's weird that Opus4 is the worst at one-shot, it requires on average two attempts to generate a valid query. If a model is really that much smarter, shouldn't it lead to better first-attempt performance? It still "thinks" beforehand, right?

Don’t talk to Opus before it’s had its coffee. Classic high-performer failure mode.

Re: Claude 4

#918

Post by jinushaun »

paulddraper wrote: post #497
bredren wrote: post #270

Earlier quoted context omitted.

It still matters for software packages. Particularly python packages that have to do with programming with AI! They are evolving quickly, with deprecation and updated documentation. Having to correct for this in system prompts is a pain. It would be great if the models were updating portions of their content more recently than others. For the tailwind example in parent-sibling comment, should absolutely be as up to d…

How often are base level libraries/frameworks changing in incomparable ways?

In the JavaScript world, very frequently. If latest is 2.8 and I’m coding against 2.1, I don’t want answers using 1.6. This happened enough that I now always specify versions in my prompt.

Re: Claude 4

#919

Post by machiaweliczny »

I personally use GPT 4.1 in simple ask mode most recently. Fast and usually correct for quite complex function so OpenAI seems to be winning IMO.

All these "agentic" things make these models so confused that it almost never gives good results in my testing.

Re: Claude 4

#920

Post by mewpmewp2 »

ModernMech wrote: post #66
jasonthorsness wrote: post #16

“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.

That's kind of my benchmark for whether or not these models are useful. I've got a project that needs some extensive refactoring to get working again. Mostly upgrading packages, but also it will require updating the code to some new language semantics that didn't exist when it was written. So far, current AI models can make essentially zero progress on this task. I'll keep trying until they can!

I think this type of thing needs agent which has access to the documentation to read about nuances of the language and package versions, definitely a way to investigate types, interfaces. Problem is that training data has so much mixed data it can easily confuse the AI to mix up versions, APIs etc.
Post reply on HN