Live data from Hacker News

Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

aavetis.github.io

111–120 of 124 posts

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#111

Earlier quoted context omitted.

Your company can be paying for it

How do I convince my company to pay for it?

It has to be an anti-pattern when all spending regardless of size effectively has to be approved by the CEO.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#113
post #110
post #100

Earlier quoted context omitted.

you'd need a pretty good opsec and non-search capable agent and logs of all its actions/chain of thought/process to be able to truly claim cleanroom implementation tho

The logs and traceability are the secret sauce here. It's one thing to have an artifact that mysteriously replicates the functionality of a well known IP-protected product without just straight up copying it. It's another thing to be able to demonstrate that said artifact was generated solely from information in the public domain or otherwise legally valid to use.

if its of your interest, i was investigating this and found out all the big labs like openai offer and indemnity clause for enterprise customers, that is supposed to assure you that it doesn't output non-compliant license code (like copyrighted or AGPL or whatever), BUT you have to accept them keeping all your logs, give them access, and let them and their lawyers do build their own case in case of getting sued.

I guess they're mostly selling insurance to bigCo's, and saying, hey we have the money to go to law, and the interests to win such a case, so we'll handle it

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#114

Earlier quoted context omitted.

Yeah sorry you got downvoted, but that's pretty much my inclination is to say RTFM. Honestly though, I'm very excited by how few developers are using the most powerful tools available. Huge opportunity for the rest of us willing to adapt to having our cheese moved, and willing to put in the work.

I do love the reaction to "here's a tool that can do everything when asked correctly" (ie, a compiler for arbitrary human artifacts), and then not read the manual. I remember a dude on this site complaining that 4o-mini only had superficial opinions of analyzing a particular poem, then it turns out the fellow didn't even supply the LLM with the text of the poem. Then the person's argument is that it is like criticizi…

I feel your pain lol just gotta let people learn on their own time I suppose

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#115

I've been underwhelmed with dedicated tools like Windsurf and Cursor in the sense that they are usually more annoying than just using ChatGPT. They have their niche but they are just so incredibly flow destroying it is hard to use them for long periods of time. I just started using Codex casually a few days ago though and already have 3 PRs. While different tools for different purposes make sense, Codex's fully async…

OpenAI nailed the UX/DX with codex. This completely obsoletes cursor and similar IDEs. I don't need AI in my tools. I just need somebody to work on my code in parallel to me. I'm happy to interact via pull requests and branches. I found out that I have access to codex on Thursday with my plus subscription. I've created and merged about a dozen PRs with it on my OSS projects since then. It's not flawless but it's pret…

I think there are basically three kinds of uses for AI: 1) "Out of loop" - e.g. Codex - it does things while you work on something else. Today it can handle basic things on its own like an appliance. 2) "In the loop" - e.g. Windsurf / Cursor. Here, you know what you are doing but are trying to use AI to essentially type at super human speeds. 3) "Coach mode" - you need to learn something in order to progress. You are using ChatGPT (usually), but possibly other tools as a way to help you get the right context faster.

Of these "In the loop", seems to be the one that doesn't work that well (yet). The main problem is latency in my opinion.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#116
post #90

All these tools seem to be GitHub-centric. Any tips for teams using GitLab to store their repositories?

I use Claude code daily at work, it writes all my PRs. It uses the GitHub cli to manage them.

Since all agents are able to use the terminal I suggest looking up the Gitlab CLI and have it use that. Should work locally and in runners.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#118
post #40

Total PRs between Codex vs Cursor is 208K vs 705, this is an enormous difference in absolute PRs. Since cursor is very popular, how does their PRs is not even 1% of codex PRs?.

This is only comparing _agents_, which is going to exclude pretty much all Cursor usage for two reasons:

* Cursor agents where just introduced in Beta and have privacy limitations that prevent their usage as many organizations.

* Cursor is still focused on hands-on-keyboard agentic flows, which aren't included in these counts.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#119

Earlier quoted context omitted.

OpenAI nailed the UX/DX with codex. This completely obsoletes cursor and similar IDEs. I don't need AI in my tools. I just need somebody to work on my code in parallel to me. I'm happy to interact via pull requests and branches. I found out that I have access to codex on Thursday with my plus subscription. I've created and merged about a dozen PRs with it on my OSS projects since then. It's not flawless but it's pret…

I think there are basically three kinds of uses for AI: 1) "Out of loop" - e.g. Codex - it does things while you work on something else. Today it can handle basic things on its own like an appliance. 2) "In the loop" - e.g. Windsurf / Cursor. Here, you know what you are doing but are trying to use AI to essentially type at super human speeds. 3) "Coach mode" - you need to learn something in order to progress. You are…

In the loop is not really a problem I have. I use intellij. So, I'm usually not really limited by my ability to type fast. I don't actually type a lot of code mostly.

A better auto complete than comes with the IDE already is actually hard and most of the AI code completion approaches I've seen conflict with the built in auto complete and don't actually don't do better. I've tried a few things and usually end up disabling the auto complete features they offer because they are quite pointless for me. What happens is that I get a lot of suggestions for code I definitely don't want drowning out the completions I do want and messing up my editing flow. Aside from having to constantly read through code that is definitely a combination of not what I'm looking for and probably wrong. And it is actually extra work that I don't need in my life. A bit of an anti feature as far as I'm concerned.

But, I actually have been using chat gpt quite a bit. It works for me because it connects to the IDE (instead of interfering with it) and it allows me to easily prompt it to ask questions about my code. This is much more useful to me than an AI second guessing me on every keystroke.

Codex adds to this by being more like a team mate that I can delegate simple things to. It would be nice if it could notify me when it is done or when it needs my input. But otherwise it's nice.

I'm pretty sure the codex and chat gpt desktop UIs might merge soon. There's no good reason to have two modalities here other than that they are probably created by two different teams. Conway's law might be an issue here. But I like what OpenAI has done with their desktop client though and they seem to be on top of that.

Re: Tracking Copilot vs. Codex vs. Cursor vs. Devin PR Performance

#120

Missing data: I don't make a codex PR if it's nonsense. Poor data: If I make one, I either if I want to: a) Merge it (success) b) Modify it (sometimes success, sometimes not). In one case, Codex made the wrong changes in all the right places, but it was still easier to work from that by hand. c) Pick ideas from it (partial success) So simple merge rates don't say much.

It isn't so much "poor" data as it is a fairly high bar for value generation. If it gets merged it is a fairly clear indicator that some value is created. If it doesn't get merged then it may be adding some value or it may not.

There's a numerator and a denominator. The numerator is fine for what you're saying -- the number of merged PRs.

The denominator varies wildly based on whether or not the PR is made. If codex makes nonsense, I don't ask it to make a PR.

Post reply on HN