Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Research Preview of Codex

openai.com

Post reply on HN
Print view View original

341–350 of 487 posts

Re: A Research Preview of Codex

#341

Post by Rastonbury »

dakiol wrote: post #230
obsolete_wagie wrote: post #226

Earlier quoted context omitted.

You need someone thats technical to look at the agent output, senior engineers will be around. Junior engineers are certainly being replaced

Thanks, Sherlock. Now, tell me, when senior engineers start to retire, who will replace them? Ah, yeah, I can hear you say "LLMs!". And LLMs will rewrite themselves so we won't need seniors anymore writing code. And LLMs will write all the code companies need. So obvious, of course. We won't need a single senior because we won't have them, because they are not hired these days anymore. Perfect plan.

Also right now, the way things are output is still constrained by an actual human engineer, junior or senior. If AI makes juniors 1.5x as effective, the company that is still investing in juniors is going to beat the competitor who decided to save costs and stop investing.

Re: A Research Preview of Codex

#342

Post by missedthecue »

boshalfoshal wrote: post #340
jetsetk wrote: post #293

Earlier quoted context omitted.

At some point the bubble will pop

Higher chance of AGI/most SWEs being out of a job for an extended period of time than the bubble popping imo.

Agree with this. It's so strange to me that out of the all the anti-AI arguments one can derive, the anti-AIers have settled on riding with the "it's useless" argument, when it's plainly one of the most useful tools created since the desktop computer.

Re: A Research Preview of Codex

#343

Post by ZeroCool2u »

"23 SWE-Bench Verified samples that were not runnable on our internal infrastructure were excluded."

What does that mean? Surely this should have a bit more elaboration. If you're just excluding a double digit number of tasks in the benchmark as uncompleted, that should be reflected in the scores.

Re: A Research Preview of Codex

#344

Post by nopinsight »

DanielVZ wrote: post #338
nopinsight wrote: post #167

Earlier quoted context omitted.

With Agentic RL training and sufficient data, AI operating at the level of average senior engineers should become plausible in a couple to a few years. Top-tier engineers who integrate a deep understanding of business and user needs into technical design will likely be safe until we get full-fledged AGI.

On the other hand I’m pretry sure you will need senior engineers not only for designing but debugging. You don’t want to hit a wall when your Agentic coder hits a bug that it just won’t fix.

There’s a recent article with experiments suggesting LLMs are better at bug fixing than coding, iirc. It’s from a company with a relevant product though.

Re: A Research Preview of Codex

#345

Post by nopinsight »

yahoozoo wrote: post #276
nopinsight wrote: post #167

Earlier quoted context omitted.

With Agentic RL training and sufficient data, AI operating at the level of average senior engineers should become plausible in a couple to a few years. Top-tier engineers who integrate a deep understanding of business and user needs into technical design will likely be safe until we get full-fledged AGI.

Why in a few years? What training data is missing that we can’t have senior level agents today?

Training data, esp interaction data from agentic coding tools, are important for that. See also: Windsurf acquisition.

Re: A Research Preview of Codex

#346

Post by sagarpatil »

johnjwang wrote: post #80

Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…

https://www.augment.new/ has a similar feature (it uses sonnet 3.7) and I’m really impressed by it. Worth trying out.

Re: A Research Preview of Codex

#347

Post by sagarpatil »

nadis wrote: post #127

In the preview video, I appreciated Katy Shi's comment on "I think this is a reflection of where engineering work has moved over the past where a lot of my time now is spent reviewing code rather than writing it." Preview video from Open AI: https://www.youtube.com/watch?v=hhdpnbfH6NU&t=878s As I think about what "AI-native" or just the future of building software loos like, its interesting to me that - right now - d…

Re:simulation Deebo does this for debugging: https://github.com/snagasuri/deebo-prototype

Re: A Research Preview of Codex

#348

Post by kypro »

prhn wrote: post #12

Is anyone using any of these tools to write non boilerplate code? I'm very interested. In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it. These tools aren't useless. They're great as search engines and pointing me i…

Firstly, LLM chat interfaces != agentic coding platforms.

ChatGPT is good for asking questions about languages, SDKs, and APIs, or generating boilerplate, but it's useless if you want to give an AI a ticket and for it to raise PRs for you.

This is where you need agentic solutions like Codex which will be far more useful because they will actually have access to your codebase and a dev environment where they can test and debug changes.

They still do really dumb things, but a lot of this can be avoided if you prompt well and give it the right types of problems to solve.

In my experience at the moment there's a sweet spot with these agentic coding platforms which makes them useful for semi-complicated tasks – assuming you prompt well they can generate 90% of the code you need, then you just need to spend the extra 10% fixing it up before it's ready for prod.

Tasks too simple (a few lines) it's a waste of time. You spend longer prompting and going back and forth with the agent than it would take to just make the change yourself.

Then obviously very complicated tasks, especially tasks that require some thought around architecture and performance, coding agents really struggle with. Less because they can't do it, but because for certain problems simply meeting ACs is far less important than how the ACs are being met. Ideally here you want to get the architecture right first, then once that's in place you can break down the remaining work for the AI to pick up.

Re: A Research Preview of Codex

#349

Post by sagarpatil »

asdev wrote: post #157

is the point of this to actually assign tasks to an AI to complete end to end? Every task I do with AI requires atleast some bit of hand holding, sometimes reprompting etc. So I don't see why I would want to run tasks in parallel, I don't think it would increase throughput. Curious if others have better experiences with this

I am working with a 3rd party API (Exa.ai) and I hacked together a python script. I ran a remote agent to do these tasks simultaneously (augment.new, I’m not affiliated, I have early access)

Agent 1: write tests, make sure all the tests pass.

Agent 2: concert python script to fastapi

Agent 3: create frontend based on fastapi endpoints

I get a PR, I check code and see if it works and then merge to main. All three PR’s worked flawlessly (front end wasn’t pretty).

Re: A Research Preview of Codex

#350

Post by sagarpatil »

yanis_t wrote: post #54

So it's looking like it's only running in the cloud, that is it will push commits to my remote repo before I have a chance to see if it works? When I'm using aider, after it make a commit what I do, I then immediately run git reset HEAD^ and then git diff (actually I use github desktop client to see the diff) to evaluate what exactly it did, and if I like it or not. Then I usually make some adjustments and only after…

It generates a PR. You decide if you want to merge, clone it locally, test, edit and then merge the code.
Post reply on HN