They seem to be injected fine in the "environment setup" but don't seem to be injected when running tasks against the enviornment. This consistently repros even if I delete and re-create the enviornment and archive and resubmit the task.
A Research Preview of Codex
281–290 of 487 posts
Re: A Research Preview of Codex
#282is the point of this to actually assign tasks to an AI to complete end to end? Every task I do with AI requires atleast some bit of hand holding, sometimes reprompting etc. So I don't see why I would want to run tasks in parallel, I don't think it would increase throughput. Curious if others have better experiences with this
codex-1 has been quite good in my experience
Re: A Research Preview of Codex
#283Earlier quoted context omitted.
If you aren't hiring junior engineers to do these kinds of things, where do you think the senior engineers you need in the future will come from? My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads. My employer has hired…
This is exactly the problem. The top level executives are setting up to retire with billions in the bank, while the workers develop their own replacements before they retire with millions in the bank. Senior developers will be mostly obsolete too. I have mentored junior developers and found it to be a rewarding part of the job. My colleagues mostly ignore juniors, provide no real guidance, couldn't care less. I see t…
Here’s a possible out: Senior engineers stop working huge corporations and use these tools to start their own businesses. (Given today’s hiring situation, this may not even be a choice.) As the business grows, hire junior developers as apprentices to handle day to day tasks while senior engineer works on bigger picture stuff. Junior engineer grows into a senior engineer who eventually uses AI to start their own business. This is a very abbreviated version of what I hope I can do, at least.
Re: A Research Preview of Codex
#284Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…
If you aren't hiring junior engineers to do these kinds of things, where do you think the senior engineers you need in the future will come from? My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads. My employer has hired…
I know this isn't what you want to hear, but what makes you think senior engineers will be in short supply in "the future"?
I'm not even a developer (anymore, I was in the past), I'm a product manager, and I'm pretty sure I can see the point in a few years where not just developers but people like me get disintermediated. My customers have a semi-reasonable grasp of what they're looking for, and they can speak. In a few years -- ten at the absolute most -- my customers will say to an AI, "I need an application that does XYZ" and the AI will reply, "Are you sure about that? Most people who say they need XYZ end up using an app that does WXY." My (former) users will reply, "Let's try it my way and see what happens." And the AI will say, "Okay, here are three popular UI styles, which do you prefer?" etc. etc.
We're headed for Interesting Times.
Re: A Research Preview of Codex
#285Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…
If you aren't hiring junior engineers to do these kinds of things, where do you think the senior engineers you need in the future will come from? My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads. My employer has hired…
I would say: ten years ago there was a huge shortage of engineers. Today, there is still lots of coding to be done, but everyone is writing code much faster and driven people learn to code to solve their own problems
Part of the reason it was so easy to get hired as a junior ten years ago was because there was so much to do. There will still be demand for engineers for a little while and then it's possible we will all be creating fairly bespoke apps and I'm not sure old me would call what future me does "programming".
Re: A Research Preview of Codex
#286[I'm one of the co-creators of SWE-bench] The team managed to improve on the already very strong o3 results on SWE-bench, but it's interesting that we're just seeing an improvement of a few percentage points. I wonder if getting to 85% from 75% on Verified is going to take as long as it took to get from 20% to 75%.
I can be completely off base, but it feels to me like benchmaxxing is going on with swe-bench. Look at the results from multi swe bench - https://multi-swe-bench.github.io/#/ swe polybench - https://amazon-science.github.io/SWE-PolyBench/ Kotlin bench - https://firebender.com/leaderboard
Re: A Research Preview of Codex
#287Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…
If you aren't hiring junior engineers to do these kinds of things, where do you think the senior engineers you need in the future will come from? My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads. My employer has hired…
On top of that, you need to be really sharp at leetcode for any large-ish company.
I find the "ai tools are junior engineers" narrative flawed, but it has any way accelerated the higher and higher expectations for a junior.
Re: A Research Preview of Codex
#288Earlier quoted context omitted.
As a dev, if you try taking away my product owners I will fight you. Who am I going to ask for requirements and sign-offs, the CEO?
Perhaps the role will merge into one, and will replace a good chunk of those jobs. E.g.: If we have 10 PMs and 90 devs today, that could be hypothetically be replace by 8 PM+Dev, 20 specialized devs, and 2 specialized PMs in the future.
Re: A Research Preview of Codex
#289Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…
If you aren't hiring junior engineers to do these kinds of things, where do you think the senior engineers you need in the future will come from? My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads. My employer has hired…
- greybeard who is trying his hardest to keep up
Re: A Research Preview of Codex
#290They mentioned "microVM" in the live stream. Notably there's no browser or internet access. It makes sense, running specialized Firecracker/Unikraft/etc microkernels is way faster and cheaper so you can scale it up. But there will be a big technical scalability difficulty jump from this to the "agents with their own computers". ChatGPT Operator already does have a browser, so they definitely can do this, but I imagin…
Demo: https://surf.e2b.dev