I think the benchmark test for these programming agents that I would like to see an Agent making a flawless PR or patch to the BSD / Linux kernel. This should be possible today and surely Linus would also see this in the future.
A Research Preview of Codex
41–50 of 487 posts
Re: A Research Preview of Codex
#42> To balance safety and utility, Codex was trained to identify and precisely refuse requests aimed at development of malicious software, while clearly distinguishing and supporting legitimate tasks. I can't say I am a big fan of neutering these paradigm-shifting tools according to one culture's code of ethics / way of doing business / etc. One man's revolutionary is another's enemy combatant and all that. What if we…
Re: A Research Preview of Codex
#43Is anyone using any of these tools to write non boilerplate code? I'm very interested. In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it. These tools aren't useless. They're great as search engines and pointing me i…
But I have done larger tasks (write device drivers) using gemini.
Re: A Research Preview of Codex
#44Just curious: is your company happy sharing their code-base with an AI provider? Or are you using a local installation?
Re: A Research Preview of Codex
#45Is there an open source version of this? that essentially uses microvms to git clone my repo and essentially run codex-cli or equivalent and sends me a PR. I made one for github action but it's not as realtime and is 2 years old now: https://github.com/asadm/chota
A not open-source option this looks close to is also https://githubnext.com/projects/copilot-workspace (released April 2024, but I'm not sure it's gotten any significant updates since)
Re: A Research Preview of Codex
#46so i just upgraded to pro plan but yet https://chatgpt.com/codex doesnt work for me and asks me to -try chatgpt pro- and shows me the upsell modal, even if already on the higher tier sigh
You mean Pro? It's only in the $200 Pro tier.
I just enabled on Settings > Connectors > Github
hoping that makes it work
... still doesnt work, is it geo-restricted maybe? idk
Re: A Research Preview of Codex
#47Maddening: "codex" is also the name of their open-source Claude-Code-alike, and was previously the name of an at-the-time frontier coding model. It's like they name things just to fuck with us.
Re: A Research Preview of Codex
#48Is anyone using any of these tools to write non boilerplate code? I'm very interested. In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it. These tools aren't useless. They're great as search engines and pointing me i…
>What am I missing? That you are trying to use LLMs to create giant sprawling codebase feature packed software packages that define the modern software landscape. What's being missed is that any one user might only utilize 5% of the code base on any given day. Software is written to accommodate every need every user could have in one package. Then the users just use the small slice that accommodates their specific ne…
But yes, I hope we get away from the giant conglomeration of everything, ESPECIALLY the reality of people doing 90% of their business inside a Google Chrome widow. Move towards the UNIX philosophy of tiny single-purpose programs.
Re: A Research Preview of Codex
#49Is it surprising? Hmm perhaps nope. But is it better than cursor etc? Hmm perhaps it’s a wrong question. Feels like codex is for product managers to fix bugs without touching any developer resources. Then it’s insanely surprising!
But now who's going to do that work? Still engineers.
Re: A Research Preview of Codex
#50I'm curious how many ICs are truly excited about these advancements in coding agents. It seems to me the general trend is we become more like PMs managing agents and reviewing PRs, all for the sake of productivity gains. I imagine many engineers are like myself in that they got into programming because they liked tinkering and hacking and implementation details, all of which are likely to be abstracted over in this n…