Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

armature.tech

Post reply on HN
Print view View original

71–80 of 175 posts

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#71

Post by antonvs »

edoceo wrote: post #29
hbarka wrote: post #20

Earlier quoted context omitted.

> Claude Code rarely searches the web while Codex almost always does I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.

Gemini CLI (at least mine) does web research all the time. I've noticed it hitting my own pages (I have to ask very specific things). I don't have any global or project rules to encourage that behavior.

That’s a deliberate effort on Google’s part. Integrating AI and search is obviously pretty critical to their business.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#72

Post by LunaSea »

hbarka wrote: post #20
screm wrote: post #2

Hey! Disclaimer: I am a Co-Founder of Armature (YC P26) which sells growth services to dev tools. This study is part of our broader work on how to influence coding agents choices and get products picked. To understand how agents pick tools we measured close to 17k sessions on an environment where agents run exactly like in the real world, on various repositories, talking to different personas (vibe-coder, junior or s…

> Claude Code rarely searches the web while Codex almost always does I’m trying to understand why they are opposite. I think it is true, I find myself giving a secondary prompt to Claude to “research this” and only then will it fetch. Codex is bang on fetching already.

OpenAI is close to Microsoft so I assume that they have preferential and cheap access to the Bing search index.

Claude is independent.

Gemini should have Google search.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#74

Post by nerdsniper »

natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

Google was pretty amazing for about 15 years, not 1 to 2. Google rocked from its launch (1999ish) until around the time it shut down Google Reader (2013ish).

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#76

Post by nucleative »

Roland303 wrote: post #61
nullbio wrote: post #58

Earlier quoted context omitted.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

how does one do that from their comfy chair?

1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of

2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools.

3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do.

4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice.

5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs.

I'm sure others will chime in, those are a few that pop the mind.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#77

Post by chupchap »

nerdsniper wrote: post #74
natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

Google was pretty amazing for about 15 years, not 1 to 2. Google rocked from its launch (1999ish) until around the time it shut down Google Reader (2013ish).

The actual reason was when they pushed Google+ and tried to make that the centre of Google. Everything else that had a social element had to be killed. Anything that breathed the same oxygen as plus got the boot.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#78

Post by pydry »

nullbio wrote: post #58
natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

If it does then software will become even more unreliable.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#79

Post by dv_dt »

LtWorf wrote: post #67
Roland303 wrote: post #61

Earlier quoted context omitted.

how does one do that from their comfy chair?

By not harassing projects with AI generated pull requests.

A pull request is a communication to the project. AI or no AI, a poorly communicated request is a burden. IMHO AI prs are fine as long as the submitter has done the work to refine that communication to make it easy to read and assess.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#80

Post by athrowaway3z »

Roland303 wrote: post #61
nullbio wrote: post #58

Earlier quoted context omitted.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

how does one do that from their comfy chair?

Unsubscribe from Anthropic because they block third party harness on subscriptions.

As long as you keep providers replaceable things will be fine.

Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.

Post reply on HN