Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harness engineering: Leveraging Codex in an agent-first world

openai.com

Post reply on HN
Print view View original

171–180 of 222 posts

Re: Harness engineering: Leveraging Codex in an agent-first world

#171

Post by aabdi »

zbrock wrote: post #170

Hello! I’m one of the three engineers who write this piece. Happy to answer questions.

Very cool article!

- are other teams adopting this approach? What’s the blockers if not?

- have there been problems where the models alone were not enough to debug and the devs had to fix it themselves?

- as the rate of changes has increased with more devs how have you dealt with concurrent writers with merge conflicts?

- if there was anything you could change in the approach you started with, what would it be?

Re: Harness engineering: Leveraging Codex in an agent-first world

#172

Post by zbrock »

h4ny wrote: post #87

I'm not an AI skeptic but I'm skeptical of the intent of this article. It makes great claims about agent-first engineering and tries to make a real case based on a real product, with real users, and a real team that's been growing — all without even saying what was built or showing it, just like every other AI hype article.

At the time we wrote the article we hadn’t released the product and weren’t ready to talk about it. It was an internal prototype that looked very much like the current Codex app.

Re: Harness engineering: Leveraging Codex in an agent-first world

#173

Post by zbrock »

cpard wrote: post #71
yurimo wrote: post #41

What I still can't understand is why is massive amount of code generated is a flex? I don't feel that software has gotten a lot better in past 3 years, only sloppier. It's surprising to me that people who know about reward hacking choose a simple objective like lines of code generated as a signal for quality. I'd argue you have to optimize for less lines generated as possible while secondary optimization should be re…

I don’t think the flex here is the amount of code alone. Their goal is to show that AI can improve productivity, the number of lines is just the proxy to that. This article is a marketing piece after all. Now someone can argue that lines of code are not a good proxy of engineering productivity, but I wouldn’t be surprised if the audience they target with this content is not the HN commenters of this thread.

Correct on the first part, partially correct on the second. LOC is a bad metric, but it is at least a legible one. Lots of people working on better ways to measure Software Productivity!

Re: Harness engineering: Leveraging Codex in an agent-first world

#175

Post by zbrock »

aabdi wrote: post #171
zbrock wrote: post #170

Hello! I’m one of the three engineers who write this piece. Happy to answer questions.

Very cool article! - are other teams adopting this approach? What’s the blockers if not? - have there been problems where the models alone were not enough to debug and the devs had to fix it themselves? - as the rate of changes has increased with more devs how have you dealt with concurrent writers with merge conflicts? - if there was anything you could change in the approach you started with, what would it be?

1. Yes! Many teams internally have adopted a lot of the same practices we outlined in the blog post. Ryan has also been spending time both internally and externally helping companies figure out how to do this in their code bases.

2. Hmm, kind of. There have definitely been issues the models can’t one shot. But we still use Codex to write all the actual code with human guidance.

3. More agents :) Some teams are experimenting with centralized Agent mediated integration queues, others use normal merge queues, many have local Codex threads that monitor CI to resolve and land conflicts or failures.

4. Today’s models and codex app. We started doing all this with gpt-5 and codex-cli. The tools today, 9 months later, are so much better than what we had then.

Re: Harness engineering: Leveraging Codex in an agent-first world

#176

Post by HorizonXP »

zbrock wrote: post #175
aabdi wrote: post #171

Earlier quoted context omitted.

Very cool article! - are other teams adopting this approach? What’s the blockers if not? - have there been problems where the models alone were not enough to debug and the devs had to fix it themselves? - as the rate of changes has increased with more devs how have you dealt with concurrent writers with merge conflicts? - if there was anything you could change in the approach you started with, what would it be?

1. Yes! Many teams internally have adopted a lot of the same practices we outlined in the blog post. Ryan has also been spending time both internally and externally helping companies figure out how to do this in their code bases. 2. Hmm, kind of. There have definitely been issues the models can’t one shot. But we still use Codex to write all the actual code with human guidance. 3. More agents :) Some teams are experi…

Have you built any tooling or products around all of this and deploying it somehow? I’d love to learn more and share notes, because we’ve been doing this too. About 3100+ PRs merged across our 4 person team in 4 months. Impossible without harness engineering, and I agree, the tools are getting even better.

Re: Harness engineering: Leveraging Codex in an agent-first world

#177

Post by chasd00 »

techblueberry wrote: post #53
bko wrote: post #2

> We had weeks to ship what ended up being a million lines of code... Five months later, the repository contains on the order of a million lines of code across application logic, infrastructure, tooling, documentation, and internal developer utilities. Over that period, roughly 1,500 pull requests have been opened and merged with a small team of just three engineers driving Codex. This translates to an average throug…

I’ve been vibe coding a lot over the past year or so, and I think I’m going to stop. In fact, I sort of want to challenge myself to see, can I go back to a sort of the fork in the road with the old copilot autocomplete workflow and really maximize that. Be in the drivers seat for most of the code being written, but find ways to use AI to really enhance the flow state / remove blockers. Tools only minimal actual code…

One workflow I like is writing a comment for what I’m about to do and then waiting a few seconds and then tab through the auto-completions. Then I check what the agent came up with, make some edits, and then on to the next block. That works well, I feel in control but don’t have to type as much.

I do use claudecode totally hands off too however. Mostly for UI tasks. Like themifying css or data grids and CRUd with all the bells and whistles, I hate that stuff and cc gets it done in minutes and mostly right. It’s also super nice to say things like “user profile in the upper right hand corner” without having to fight css.

/if it’s not clear, I hate dealing with css and related frameworks.

Re: Harness engineering: Leveraging Codex in an agent-first world

#178

Post by endomorphosis »

bko wrote: post #2

> We had weeks to ship what ended up being a million lines of code... Five months later, the repository contains on the order of a million lines of code across application logic, infrastructure, tooling, documentation, and internal developer utilities. Over that period, roughly 1,500 pull requests have been opened and merged with a small team of just three engineers driving Codex. This translates to an average throug…

I have been building an entire operating system ( not figuratively)

Prior to ai autocomplete 500 loc a day and then with ai autocomplete I could do 2500 a day and now 50k is pretty normal. Walking around tech week with my phone yielded 150k this week

Re: Harness engineering: Leveraging Codex in an agent-first world

#179

Post by HorizonXP »

bko wrote: post #2

> We had weeks to ship what ended up being a million lines of code... Five months later, the repository contains on the order of a million lines of code across application logic, infrastructure, tooling, documentation, and internal developer utilities. Over that period, roughly 1,500 pull requests have been opened and merged with a small team of just three engineers driving Codex. This translates to an average throug…

[flagged]

Re: Harness engineering: Leveraging Codex in an agent-first world

#180

Post by chasd00 »

59nadir wrote: post #121
techblueberry wrote: post #53

Earlier quoted context omitted.

I’ve been vibe coding a lot over the past year or so, and I think I’m going to stop. In fact, I sort of want to challenge myself to see, can I go back to a sort of the fork in the road with the old copilot autocomplete workflow and really maximize that. Be in the drivers seat for most of the code being written, but find ways to use AI to really enhance the flow state / remove blockers. Tools only minimal actual code…

I would be very impressed with someone who's been vibecoding "a lot" for about a year who could then go back to being fully in the loop for even 50%. I would even say I'd expect withdrawal symptoms at that point. The dopamine hits are core to why people even do vibecoding (or vibecoding-in-a-dress/spec-driven development) and why they tend to overestimate its output so much. Hell, it's core to all forms of LLM-assist…

The dopamine hit is real, I feel like that was identified early on by OpenAI and probably lit a fire to get ChatGPT in the hands of the public. Bf Skinner (I think) is the one who narrowed in on variable ratio reward systems to maximize operant conditioning. An LLM, with hallucinations and imperfections, is the perfect variable ratio reward system. It’s no wonder they’re getting pushed so hard along with a consumption based pricing model. Whether you’re a human, rat, plant, bacteria there’s no real defense against that kind of conditioning.

First hit on Google

https://www.simplypsychology.org/operant-conditioning.html

Post reply on HN