Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

aavetis.github.io

Post reply on HN
Print view View original

81–90 of 124 posts

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#81

Post by tough »

cap11235 wrote: post #35
elliotec wrote: post #28

Earlier quoted context omitted.

Have you tried Claude code? I’m surprised it’s not in this analysis but in my personal experience, the competition doesn’t even touch it. I’ve tried them all in earnest. My toolkit has been (neo)vim and tmux for at least a decade now so I understand the apprehension for less terminal-inclined folks that prefer other stuff but it’s my jam and just crushes it.

Right, after the Sonnet 4 release it was the first time I could tell an agent something and just let it run comfortably. As for the tool itself, I think a large part of its ability comes from how it writes recursive todo-lists for itself, which are shown to the user, so you can intervene early on the occasions it goes full Monkey's Paw.

yeah i've been manually doing first a TASKS.md so i can modify it while the agent starts working on it.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#83

Post by rustc »

polskibus wrote: post #57
lukehoban wrote: post #21

(Disclaimer: I work on coding agents at GitHub) This data is great, and it is exciting to see the rapid growth of autonomous coding agents across GitHub. One thing to keep in mind regarding merge rates is that each of these products creates the PR at a different phase of the work. So just tracking PR create to PR merge tells a different story for each product. In some cases, the work to iterate on the AI generated co…

What is your team’s take on the copyright for commits generated by ai agent ? Would the copyright protect it? Current US stance seems to be: https://www.copyright.gov/newsnet/2025/1060.html “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements”. If entire commit is generated by AI then it is obvious what created it - it’s…

> “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements”

How would that work if it's a patch to a project with a copyleft license like GPL which requires all derivate work to be licensed the same?

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#84

Post by anticensor »

rustc wrote: post #83
polskibus wrote: post #57

Earlier quoted context omitted.

What is your team’s take on the copyright for commits generated by ai agent ? Would the copyright protect it? Current US stance seems to be: https://www.copyright.gov/newsnet/2025/1060.html “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements”. If entire commit is generated by AI then it is obvious what created it - it’s…

> “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements” How would that work if it's a patch to a project with a copyleft license like GPL which requires all derivate work to be licensed the same?

GPL is a copyright licence, not a ToS.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#85

Post by rustc »

anticensor wrote: post #84
rustc wrote: post #83

Earlier quoted context omitted.

> “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements” How would that work if it's a patch to a project with a copyleft license like GPL which requires all derivate work to be licensed the same?

GPL is a copyright licence, not a ToS.

> GPL is a copyright licence, not a ToS.

How is ToS relevant to this thread?

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#86

Post by cap11235 »

mmaunder wrote: post #74
cap11235 wrote: post #65

Earlier quoted context omitted.

Or you stick it in docker. Or actually configure your permissions. RTFM

Yeah sorry you got downvoted, but that's pretty much my inclination is to say RTFM. Honestly though, I'm very excited by how few developers are using the most powerful tools available. Huge opportunity for the rest of us willing to adapt to having our cheese moved, and willing to put in the work.

I do love the reaction to "here's a tool that can do everything when asked correctly" (ie, a compiler for arbitrary human artifacts), and then not read the manual. I remember a dude on this site complaining that 4o-mini only had superficial opinions of analyzing a particular poem, then it turns out the fellow didn't even supply the LLM with the text of the poem. Then the person's argument is that it is like criticizing someone for their hammer being 2.7mm off center. Utterly ridiculous; LLMs are not psychic, they just have approximate knowledge of many things. People seem to love setting them up to fail. My favorite "demonstration" is showing LLMs messing up doing multiplication of large numbers. If only the LLMs had access to some sort of machine that could do multiplication well...

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#87

Post by olq_plo »

rustc wrote: post #83
polskibus wrote: post #57

Earlier quoted context omitted.

What is your team’s take on the copyright for commits generated by ai agent ? Would the copyright protect it? Current US stance seems to be: https://www.copyright.gov/newsnet/2025/1060.html “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements”. If entire commit is generated by AI then it is obvious what created it - it’s…

> “It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements” How would that work if it's a patch to a project with a copyleft license like GPL which requires all derivate work to be licensed the same?

IANAL, but it means the commit itself is public domain. When integrated into a code base with a more restrictive license, you can still use that isolated snippet in whatever way you want.

More interesting question is whether one could remove the GPL restrictions on public code by telling AI to rewrite the code from scratch, providing only the behavior of the code.

This could be accomplished by making AI generate a comprehensive test suite first, and then let it write the code of the app seeing only the test suite.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#88

Post by cap11235 »

HenriNext wrote: post #79
ainiriand wrote: post #66

Earlier quoted context omitted.

It is not an 'agent' in the sense that it is not really autonomous afaik.

Claude Code was not designed from the ground up to be only an autonomous agent, but it can certainly act as one. - It has non-interactive CLI functionality (with -p "prompt" option) in addition to the default interactive TUI, making it easy to integrate to workflows. - It has turn-key GitHub integration ( https://github.com/anthropics/claude-code-action ). - It has internal task tracking system that uses ReadTodo/Wri…

Yeah, my primary usage pattern for it is purely autonomous for new feature development. I have Claude iterate on a prompt for itself a lot, asking me questions as it goes, then after, I can just say generic things like "Do the thing", "Continue", "Check the repo" and it does the thing, based on R/W Todo and my larger scale todo list for implementation. Also, Claude does have a github action (not that I've tried it though).

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#89

Post by falcor84 »

cap11235 wrote: post #46
ezyang wrote: post #44

Earlier quoted context omitted.

The happy path way of getting code out of Codex is a PR. This is emphatically not true for Cursor.

Feels like a sort of pollution.

Why? That is its intent - unlike an IDE, it is intended to work autonomously and only get back to you after it has prepared the full changeset - which at that point you'd review via a PR. Where's the pollution in that?
Post reply on HN