Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4 System Card

simonwillison.net

Post reply on HN
Print view View original

111–120 of 264 posts

Re: Claude 4 System Card

#111

Post by trip-zip »

Workaccount2 wrote: post #108
pauldix wrote: post #98

Earlier quoted context omitted.

My experience so far with Opus 4 is that it's very good. Based on a few days of using it for real work, I think it's better than Sonnet 3.5 or 3.7, which had been my daily drivers prior to Gemini 2.5 Pro switching me over just 3 weeks ago. It has solved some things that eluded Gemini 2.5 Pro. Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable. B…

I've been having really good results from Jules, which is Google's gemini agent coding platform[1]. In the beta you only get 5 tasks a day, but so far I have found it to be much more capable than regular API Gemini. [1] https://jules.google/

Would you mind giving a little more info on what you're getting Jules to work on? I tried it out a couple times but I think I was asking for too large a task and it ended up being pretty bad, all things considered.

I tried to get it to add some new REST endpoints that follow the same pattern as the other 100 we have, 5 CRUD endpoints. It failed pretty badly, which may just be an indictment on our codebase...

Re: Claude 4 System Card

#112

Post by Balgair »

BoppreH wrote: post #86

> This includes locking users out of systems that it has access to or bulk-emailing media and law-enforcement figures to surface evidence of wrongdoing. Isn't that a showstopper for agentic use? Someone sends an email or publishes fake online stories that convince the agentic AI that it's working for a bad guy, and it'll take "very bold action" to bring ruin to the owner.

Yeah, I mean that's likely not what 'individual persons' are going to want.

But Holy shit, that exactly what 'people' want. Like, when I read that, my heat was singing. Anthropic has a modicum of a chance here, as one of the big-boy AIs, to make an AI that is ethical.

Like, there is a reasonable shot here that we thread the needle and don't get paperclip maximizers. It actually makes me happy.

Re: Claude 4 System Card

#113

Post by simonw »

I just published a deep dive into the Claude 4 system prompts, covering both the ones that Anthropic publish and the secret tool-defining ones that got extracted through a prompt leak. They're fascinating - effectively the Claude 4 missing manual: https://simonwillison.net/2025/May/25/claude-4-system-prompt...

Re: Claude 4 System Card

#114

Post by cleak »

smokel wrote: post #110
pauldix wrote: post #98

Earlier quoted context omitted.

My experience so far with Opus 4 is that it's very good. Based on a few days of using it for real work, I think it's better than Sonnet 3.5 or 3.7, which had been my daily drivers prior to Gemini 2.5 Pro switching me over just 3 weeks ago. It has solved some things that eluded Gemini 2.5 Pro. Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable. B…

> Gemini's 1M token context window is really unbeatable. How does that work in practice? Swallowing a full 1M context window would take in the order of minutes, no? Is it possible to do this for, say, an entire codebase and then cache the results?

I’m curious about this as well, especially since all coding assistants I’ve used truncate long before 1M tokens.

Re: Claude 4 System Card

#115

Post by danielbln »

markovs_gun wrote: post #69
torginus wrote: post #63

Earlier quoted context omitted.

Man I miss Claude 2. It talked like a competent, but incredibly lazy person who didn't care for formality and wanted to get the interaction over with in the shortest possible time.

That's exactly what I want from an LLM. But then again I want a tool and not a robot prostitute

Gemini is closer to that, imo, especially when calling the API. It pushes back more and doesn't do as much of the "That's brilliant!" dance.

Re: Claude 4 System Card

#116

Post by pauldix »

smokel wrote: post #110
pauldix wrote: post #98

Earlier quoted context omitted.

My experience so far with Opus 4 is that it's very good. Based on a few days of using it for real work, I think it's better than Sonnet 3.5 or 3.7, which had been my daily drivers prior to Gemini 2.5 Pro switching me over just 3 weeks ago. It has solved some things that eluded Gemini 2.5 Pro. Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable. B…

> Gemini's 1M token context window is really unbeatable. How does that work in practice? Swallowing a full 1M context window would take in the order of minutes, no? Is it possible to do this for, say, an entire codebase and then cache the results?

Right now this is just in the AI Studio web UI. I have a few command line/scripts to put together a file or two and drop those in. So far I've put in about 450k of stuff there and then over a very long conversation and iterations on a bunch of things built up another 350k of tokens into that window.

Then start over again to clean things out. It's not flawless, but it is surprising what it'll remember from a while back in the conversation.

I've been meaning to pick up some of the more automated tooling and editors, but for the phase of the project I'm in right now, it's unnecessary and the web UI or the Claude app are good enough for what I'm doing.

Re: Claude 4 System Card

#117

Post by mountainriver »

Closi wrote: post #71
aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

> Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? My experience is the opposite - I'm using it in Cursor and IMO it's performing better than Gemini 2.5 Pro at being able to write code which will run first time (which it wasn't before) and seems to be able to complete much la…

It’s funny how differently the models work in cursor. Claude 4 thinks then takes one little step at a time, but yes it’s quite good overall

Re: Claude 4 System Card

#118

Post by ZeroCool2u »

smokel wrote: post #110
pauldix wrote: post #98

Earlier quoted context omitted.

My experience so far with Opus 4 is that it's very good. Based on a few days of using it for real work, I think it's better than Sonnet 3.5 or 3.7, which had been my daily drivers prior to Gemini 2.5 Pro switching me over just 3 weeks ago. It has solved some things that eluded Gemini 2.5 Pro. Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable. B…

> Gemini's 1M token context window is really unbeatable. How does that work in practice? Swallowing a full 1M context window would take in the order of minutes, no? Is it possible to do this for, say, an entire codebase and then cache the results?

In my experience with Gemini it definitely does not take a few minutes. I think that's a big difference between Claude and Gemini. I don't know exactly what Google is doing under the hood there, I don't think it's just quantization, but it's definitely much faster than Claude.

Caching a code base is tricky, because whenever you modify the code base, you're invalidating parts of the cache and due to conditional probability any changed tokens will change the results.

Re: Claude 4 System Card

#119

Post by jjbinx007 »

simonw wrote: post #113

I just published a deep dive into the Claude 4 system prompts, covering both the ones that Anthropic publish and the secret tool-defining ones that got extracted through a prompt leak. They're fascinating - effectively the Claude 4 missing manual: https://simonwillison.net/2025/May/25/claude-4-system-prompt...

Truly fascinating, thanks for this.

What I find a little perplexing is when AI companies are annoyed that customers are typing "please" in their prompts as it supposedly costs a small fortune at scale yet they have system prompts that take 10 minutes for a human to read through.

Re: Claude 4 System Card

#120

Post by [deleted] »

Workaccount2 wrote: post #108
pauldix wrote: post #98

Earlier quoted context omitted.

My experience so far with Opus 4 is that it's very good. Based on a few days of using it for real work, I think it's better than Sonnet 3.5 or 3.7, which had been my daily drivers prior to Gemini 2.5 Pro switching me over just 3 weeks ago. It has solved some things that eluded Gemini 2.5 Pro. Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable. B…

I've been having really good results from Jules, which is Google's gemini agent coding platform[1]. In the beta you only get 5 tasks a day, but so far I have found it to be much more capable than regular API Gemini. [1] https://jules.google/

[deleted]
Post reply on HN