Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

armature.tech

Post reply on HN
Print view View original

171–175 of 175 posts

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#171

Post by Neywiny »

reneeh wrote: post #169
Neywiny wrote: post #168

Earlier quoted context omitted.

Because they're thinking like I did going into the article. Harnesses like Claude expose "tools" to the agent. I usually use Cline but I'm giving up on it for this exact reason. Cline tells the model "you tell me to write a file, I'll get it done" and then it messes everything up, causes tones of errors, and the model goes "wow that's a broken tool. I'm going to write a python script to write the file instead"

Cline just recently fully upgraded their harness, see here: https://x.com/cline/status/2095897914493243512?s=20 Try if you have a better experience now!

They said as of aug 23rd it's 100% rollout. I've noticed 0 improvement. If anything a downgrade. The agents often refuse to use the tools after 1 try because the results are so trash. An open ticket that shows my biggest gripe with the write tool https://github.com/cline/cline/issues/13276

I guess the mistake count is down though. But that's because it's bypassing all the tools and just running commands

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#172

Post by pixelatedindex »

pixelatedindex wrote: post #63
Roland303 wrote: post #61

Earlier quoted context omitted.

how does one do that from their comfy chair?

I for one am running what I can on my aging 1080Ti(!!), namely a Qwen2.5 14B (4-bit quantized). It’s not great, and the only other card I have is a 3070Ti but I need that for gaming. :( What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.

I really don’t understand why this is being downvoted? Did I say something wrong / controversial? Or is it because I’m not using my resources the “optimal” way? Is it because my models aren’t open weight? I was only trying to contribute my experience :(

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#173

Post by lucas_t_a »

ttul wrote: post #25

I built this for my own company. Armature is on to something. You start by analyzing the choices agents would make for various use cases and then glean what, if anything, you might do to start tilting the agents in the direction of your own product and away from the competitor. Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner f…

that's just making a good and complete product then

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#174

Post by kyleaa »

nullbio wrote: post #58
natnatenathan wrote: post #56

I keep telling people that we are living in the golden age of AI - like the first year or two of google. It is all down hill as these companies push for profit and lock-in.

Exactly why everyone needs to be hyper-focused on ensuring that the open-source ecosystem is healthy and that we don't let them shut that down.

Does Nvidia's acquisition put the open-source ecosystem at risk? I remember what Oracle purchased Java. IMO, it was a mess for many years after that. It is a strategic move for Nvidia, but I see it harming open-source AI options.

Re: Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

#175

Post by distances »

sajithdilshan wrote: post #120
Pavilion2095 wrote: post #114

Earlier quoted context omitted.

In many countries Meta platforms—Facebook, Instagram, WhatsApps—are primary means of communication. If you want to find interesting events happening this weekend, you need to be on Facebook. In this regard, Google and Apple are much easier to leave because they offer personalised services, and you can always move your email and photos elsewhere. But you cannot move other people from Facebook or Instagram.

It’s not that easy to leave Google or Apple. As an example I wanted to leave iCloud and stop paying storage tax for Apple. But they have made it almost impossible to migrate the photos. There is no API to download the photos programmatically and the only way is to sync all the photos to MacBook and then move it. However, the photos are close to 1TB of storage and my MacBook doesn’t have enough storage to sync all tho…

> I have used my Gmail address for countless services and it’s almost part of my official identity.

This has been a topic for over a decade now, I think. People have been recommending getting your own domain. Best time to do that was ten years ago, and the second best is today.

Post reply on HN