Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

armature.tech

Post reply on HN
Print view View original

101–110 of 175 posts

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#101

Post by olmo23 »

xnorswap wrote: post #98

They didn't measure which programming language because we all know it's python, which it tries to use, every session, despite repeated memory files to not use python. Recently it's even taken to installing python to get jobs done.

Perhaps it depends on your previously saved memories, because over here if given the choice it's always reaching for either TypeScript or C.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#102

Post by speedgoose »

natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

You can run pretty good LLMs locally. Currently the hardware is expensive but it will be cheaper eventually.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#104

Post by sjducb »

troupo wrote: post #83
nullbio wrote: post #58

Earlier quoted context omitted.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

> on ensuring that the open-source ecosystem There are no open source models. Only open weights . No one is giving you the source (training data). And yeah, no one is giving you the compute to train the models.

I think OP means the existing open source ecosystem, things like the Python requests library.

There’s a real risk that AI kills contributions to traditional open source projects, and we all move to custom libraries written by our own AI.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#105

Post by m12k »

natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

I haven't used Kimi K3 or GLM 5.3, but boy am I glad they exist! Every time an open weights model is released a bit closer to the frontier models, that's another level of performance that we, the users, can't lose access to no matter if the frontier labs jack up prices out of our reach, or the US administration decides to put a full export block in place on AI.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#107

Post by nullbio »

troupo wrote: post #83
nullbio wrote: post #58

Earlier quoted context omitted.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

> on ensuring that the open-source ecosystem There are no open source models. Only open weights . No one is giving you the source (training data). And yeah, no one is giving you the compute to train the models.

Yes, there are. But that's not what I was saying. I'm talking about harnesses, tooling, sharing training data, the shared research, and the list goes on. Frontier labs are already trying to move everyone into the cloud so their harnesses and surrounding tooling can be hidden away. We need to make sure there are strong open-source competitors to this model that keep the power on the local machine.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#108

Post by screm »

paulhebert wrote: post #34
folkrav wrote: post #31

Earlier quoted context omitted.

I feel like you legitimately cannot say it for legal reasons, but I wish we would just name these freaking things.

Didn’t OpenAI claim they’re going to make a billion dollars off ads in a year or something?

True but they'll probably never integrate ads into the model's thinking (for now it's only a separate display in the apps). When tokens are a commodity you'll just switch to the one you can trust and since all labs are soon going to become tokenmeter companies (Sam's own words), I don't think they can afford to do that.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#109

Post by tuberreact »

I appreciate the eval design here. And man Claude not doing web searches is killing me bc it just won't offer the most up to date information. At the same time, there's research saying Claude relies heavily on Brave search so I'm not sure how to reconcile

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#110

Post by screm »

tuberreact wrote: post #109

I appreciate the eval design here. And man Claude not doing web searches is killing me bc it just won't offer the most up to date information. At the same time, there's research saying Claude relies heavily on Brave search so I'm not sure how to reconcile

If you want your Claude Code to search you can always tweak your own with a good skill, this should work perfectly! It's more a problem for vendors who can't tell all people in the world to download a specific skill first.
Post reply on HN