Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harness engineering: Leveraging Codex in an agent-first world

openai.com

Post reply on HN
Print view View original

121–130 of 222 posts

Re: Harness engineering: Leveraging Codex in an agent-first world

#121

Post by 59nadir »

techblueberry wrote: post #53
bko wrote: post #2

> We had weeks to ship what ended up being a million lines of code... Five months later, the repository contains on the order of a million lines of code across application logic, infrastructure, tooling, documentation, and internal developer utilities. Over that period, roughly 1,500 pull requests have been opened and merged with a small team of just three engineers driving Codex. This translates to an average throug…

I’ve been vibe coding a lot over the past year or so, and I think I’m going to stop. In fact, I sort of want to challenge myself to see, can I go back to a sort of the fork in the road with the old copilot autocomplete workflow and really maximize that. Be in the drivers seat for most of the code being written, but find ways to use AI to really enhance the flow state / remove blockers. Tools only minimal actual code…

I would be very impressed with someone who's been vibecoding "a lot" for about a year who could then go back to being fully in the loop for even 50%. I would even say I'd expect withdrawal symptoms at that point.

The dopamine hits are core to why people even do vibecoding (or vibecoding-in-a-dress/spec-driven development) and why they tend to overestimate its output so much. Hell, it's core to all forms of LLM-assisted development (because it feels like magic), but most of the other forms are more value, less delusion.

Re: Harness engineering: Leveraging Codex in an agent-first world

#122

Post by pramodbiligiri »

ajpaulson wrote: post #75

> The diagram below shows the rule: within each business domain (e.g. App Settings), code can only depend “forward” through a fixed set of layers (Types → Config → Repo → Service → Runtime → UI). Cross-cutting concerns (auth, connectors, telemetry, feature flags) enter through a single explicit interface: Providers. Anything else is disallowed and enforced mechanically. Can anyone give me a simplified explanation of…

As other commenters have clarified, it's about layering, separation of concerns etc. Goes by many names. One such terminology here: https://en.wikipedia.org/wiki/Hexagonal_architecture_(softwa.... DI frameworks use terminology like "Provider": https://en.wikipedia.org/wiki/Dependency_injection#Injectors

Re: Harness engineering: Leveraging Codex in an agent-first world

#123

Post by mohsen1 »

I've been doing the same experiment in tsz[1] for a while now (the same past five months in fact) and I have come to very similar conclusions. Lots of harness to enforce good architecture splits. Lots of tests and CI.

My point of working on tsz is to learn how to do very big projects with AI. Eventually the same workflows and attitude can be leveraged to build customer product apps with UI as well. I see that OpenAI is leveraging automated browser testing and even videos as part of their workflow. I think as models get better this direction for making software would eventually make sense. I don't think we're there yet though. But at least, unlike OpenAI vague claims I can share the output with you to see!

Most of the solutions that offer a very high level of automation like Lovable are a bit too optimistic and solutions are not tightly coupled with lots of automated testing.

[1] https://github.com/tsz-org/tsz

Re: Harness engineering: Leveraging Codex in an agent-first world

#125

Post by sajithdilshan »

charintstr wrote: post #52

I am at a major company that is essentially vibe coding. I’ve shipped about 100k LoC this entire half and am toward top 10% of my team. I find it likely that either A. The code is absolute garbage and is speed for speed sake B. They’re using an internal model that is a generation beyond GPT 5.5 I say this because we’ve attempted to do something similar using the latest gen Claude models and a significantly larger tea…

I think vibe coding is okay for a small internal tool, dashboard, etc. but it’s definitely a no-go for a production running service.

Re: Harness engineering: Leveraging Codex in an agent-first world

#126

Post by aniceperson »

andai wrote: post #27

> To drive a PR to completion, we instruct Codex to review its own changes locally, request additional specific agent reviews both locally and in the cloud, respond to any human or agent given feedback, and iterate in a loop until all agent reviewers are satisfied (effectively this is a Ralph Wiggum Loop ). https://ghuntley.com/loop/

ok, but you had 1x token to generate , then more 1x to review locally, 1x for the agent local, 1 x for the cloud. then ???x until all bots are satisfied.

You end-up spending at least 5x the amount of tokens for maybe prediction machine to find a discontinuity?

I would say a way better approach is 1.123x to generate code + tests + passing analysis tools + human review + 1x "simplify as much as possible", than letting the snake its own tail without boundaries.

Re: Harness engineering: Leveraging Codex in an agent-first world

#127

Post by small_model »

yurimo wrote: post #41

What I still can't understand is why is massive amount of code generated is a flex? I don't feel that software has gotten a lot better in past 3 years, only sloppier. It's surprising to me that people who know about reward hacking choose a simple objective like lines of code generated as a signal for quality. I'd argue you have to optimize for less lines generated as possible while secondary optimization should be re…

It's not massive amount of code for the sake of it, it's that it built a complex large piece of software itself. If you compared the linux kernel LOC to some guys toy kernel on GitHub the LOC difference should give you an idea on the complexity. "Yeah why is Tolstoy flexing about his number of pages in his book, I wrote Spot goes to school and its only 20 pages"

Re: Harness engineering: Leveraging Codex in an agent-first world

#128

Post by small_model »

jh00ker wrote: post #113
yurimo wrote: post #41

What I still can't understand is why is massive amount of code generated is a flex? I don't feel that software has gotten a lot better in past 3 years, only sloppier. It's surprising to me that people who know about reward hacking choose a simple objective like lines of code generated as a signal for quality. I'd argue you have to optimize for less lines generated as possible while secondary optimization should be re…

>I suspect it's not seen as a problem by providers because more lines generated means more tokens used and hence more billing put out on customers. I have also grown skeptical of token usage in order to run up my bill! But since I feel like it takes me MORE effort to write LESS lines of code myself, I'd expect a quick and dirty AI-generated solution to be MORE lines of code and cost LESS to generate than a concise/el…

The latest frontier models will write code better than you and more elegant, with less lines of code, in 100th of the time, with full test coverage. Hand coding is like writing out assembly/machine code rather than using a compiler.

Re: Harness engineering: Leveraging Codex in an agent-first world

#130

Post by csomar »

YZF wrote: post #73
yurimo wrote: post #41

What I still can't understand is why is massive amount of code generated is a flex? I don't feel that software has gotten a lot better in past 3 years, only sloppier. It's surprising to me that people who know about reward hacking choose a simple objective like lines of code generated as a signal for quality. I'd argue you have to optimize for less lines generated as possible while secondary optimization should be re…

Lines of code has always been a terrible metric. But all else being equal it is a measure. If all else is not equal, which is usually the case, then it's not. A lot of the focus has been on AI recently. Three years ago we didn't have software where a non-software engineer can describe what they want in English and get working (-ish) software generated by other software? Is that not "software has gotten a lot better"?…

> Is Google Maps suddenly taking you the wrong way more often?

Funny you mention that because I had that issue in a cab just yesterday. Google decided to drive us of the main road to a series of small roads which happened to be a dead end. My guess is that the AI decided that this is a shorter road? less busier road?

That being said, Google maps have been gradually degrading. Most notably, its search function is quasi-broken now.

Post reply on HN