Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harness engineering: Leveraging Codex in an agent-first world

openai.com

Post reply on HN
Print view View original

61–70 of 222 posts

Re: Harness engineering: Leveraging Codex in an agent-first world

#61

Post by therealdrag0 »

zatkin wrote: post #23

I worry most about blindspots with this kind of approach. Let's say that this repository goes on for years, at which point the docs folder is several MB in size. Would Codex be able to think outside of the box? Or would the aggregate of the Markdown content fundamentally cover enough ground to prevent it from thinking of novel new approaches to existing problems?

It’s not a self coding machine. There is human in the loop, they even added MORE engineers to the team of this project! 7 engineers should be able to collaborate with the AI to find good solutions to problems.

Re: Harness engineering: Leveraging Codex in an agent-first world

#62

Post by drivebyhooting »

I wish these breathless blog posts would actually try to be more didactic.

For example, actually doing a walkthrough of how to set up these allegedly super powered workflows and concrete demonstrations.

I’m not an AI skeptic. Rather I’d don’t want to miss out on any actual super powers.

Re: Harness engineering: Leveraging Codex in an agent-first world

#63

Post by therealdrag0 »

charintstr wrote: post #52

I am at a major company that is essentially vibe coding. I’ve shipped about 100k LoC this entire half and am toward top 10% of my team. I find it likely that either A. The code is absolute garbage and is speed for speed sake B. They’re using an internal model that is a generation beyond GPT 5.5 I say this because we’ve attempted to do something similar using the latest gen Claude models and a significantly larger tea…

I like how they said they were spending 20% of their time addressing slop. Sounds like they’ve tried to automate the slop correction but it’s a good honest reminder.

Additionally it’s an internal tool, which is likely much more amenable to slop.

Re: Harness engineering: Leveraging Codex in an agent-first world

#64

Post by Frannky »

I started using chatgpt for functions and checking, then for single file changes and checking, now for multiple changes and checking. I am at a point where the only changes I correct are architectural. So it may start to become smarter to learn how to see only the architectural directions while multiple agents work, test, and commit both on unit and against live deployment.

Re: Harness engineering: Leveraging Codex in an agent-first world

#65

Post by thelucent »

This might work only if you have “infinite” compute and infinite tokens.

As someone that used the $20 plan, this pure agentic approach is impossible to do because I’d hit the limit fast and I would end up with less outcome.

What I found that work incredibly well was to provide a human written code as reference, and ask it to extend it. So I scaffold the entire thing, architect it, write few samples code (controllers, services, models, components, database schema, how auth works, etc) so the LLM can have a headstart on their attention (pun intended)

I usually wrote a stub with a lot of details on how to implement it. Something like a higher abstraction pseudo code. Then ask the LLM to implement it.

When it fails, it is often better to undo the whole changes, adjust the stub so it catches what fails before, and try again.

Or, commit the changes, and use a new fresh context and only address what went wrong.

-

Whenever I tried this agentic from scratch approach, I always end up disappointed; both on the outcome and on the limit that I hit before an hour even passed.

Re: Harness engineering: Leveraging Codex in an agent-first world

#68

Post by nimonian »

drivebyhooting wrote: post #62

I wish these breathless blog posts would actually try to be more didactic. For example, actually doing a walkthrough of how to set up these allegedly super powered workflows and concrete demonstrations. I’m not an AI skeptic. Rather I’d don’t want to miss out on any actual super powers.

I do quite a lot of what this post describes in a reasonably large project. Here's what works for me:

- write gherkin features for new features; update them for enhancements; don't touch them for refactors. Label your PRs with these nouns.

- use pre-push hooks for type checks, linting, unit tests, and other quick, scriptable validations.

- make a viteperess subsite in your repo, have the agents maintain it - document important principles, architecture, etc.

- make a cli command which lists all pages along with the yaml frontmatter description so agents can choose what to read without blowing up the context window.

- use ddd and monorepo - write your logic in headless layers, and compose layers into apps. agents navigate layers very successfully.

- use zod (or your language equivalent) and contract-first API development; this is my favourite bit tbh, I use orpc

- make a single skill called "code" which describes the lifecycle: open a worktree, setup .env to guarantee no conflict with other agents (choose unused ports etc - docker is good here), write or update feature file (this is where you negotiate the spec), implement, validate (e.g. using playwright mcp), pre-push checks, push and wait for review, tear down and fast forward main

- testcontainers is great for ensuring multiple agents can run tests that don't conflict

Seriously I only have one skill that's it. Everything else is in the docs. I'm feeling very productive like this, in a "making good software" sense not a LoC sense.

Re: Harness engineering: Leveraging Codex in an agent-first world

#69

Post by trollbridge »

yurimo wrote: post #41

What I still can't understand is why is massive amount of code generated is a flex? I don't feel that software has gotten a lot better in past 3 years, only sloppier. It's surprising to me that people who know about reward hacking choose a simple objective like lines of code generated as a signal for quality. I'd argue you have to optimize for less lines generated as possible while secondary optimization should be re…

The "lines of code" at this point are basically the same thing as binary code that comes out of a compiler - something you almost never look at and certainly won't try to touch by hand.

The actual "code" is everything driving the harness.

The current problem for this is that the harness is not (yet) deterministic, so it's sort of like having a compiler where your output program works slightly differently every build, and then the compiler tries to just patch the binary programs when you recompile to minimise this problem, or even worse, disassembles the whole thing to figure out what it does, makes the chance, and then recompiles it.

Post reply on HN