Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Research Preview of Codex

openai.com

Post reply on HN
Print view View original

81–90 of 487 posts

Re: A Research Preview of Codex

#81

Post by GolfPopper »

btbuildem wrote: post #31

> To balance safety and utility, Codex was trained to identify and precisely refuse requests aimed at development of malicious software, while clearly distinguishing and supporting legitimate tasks. I can't say I am a big fan of neutering these paradigm-shifting tools according to one culture's code of ethics / way of doing business / etc. One man's revolutionary is another's enemy combatant and all that. What if we…

>What if we need top-notch malware to take down the robot dogs lobbing mortars at our madmaxian compound?!

I wouldn't sweat it. According to it's developers, Codex understands 'malicious software', it has just been trained to say, "But I won't do that" when such requests are made to it. Judging from the recent past [1][2] getting LLMs to bypass such safeguards is pretty easy.

1.https://hiddenlayer.com/innovation-hub/novel-universal-bypas... 2.https://cyberpress.org/researchers-bypass-safeguards-in-17-p...

Re: A Research Preview of Codex

#83

Post by IXCoach »

IXCoach wrote: post #79
prhn wrote: post #12

Is anyone using any of these tools to write non boilerplate code? I'm very interested. In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it. These tools aren't useless. They're great as search engines and pointing me i…

Hey there! Lots missing here, but I had the same issues, it takes iteration and practice. I use claude code in terminal windows, and text expander to save explicit reminders that I have to inject super regularly because anthropic obscures access to system prompts. For example, I have 3 to 8 paragraph long instructions I will place regularly about not assuming, checking deterministically etc. and for most things I hav…

Its a total of about 30 snippets, avg 6 paragraphs long, that I have to inject. for each role switch it goes through i have to re inject them.

its a pain but it works.

Even TDD it will hallucinate the mocks without management. and hallucinate the requirements. Each layer has to be checked atomically, but the text expander snippets done right can get it close to 75% right.

My main project faces 5000 users so I cant let the agents run freely, whereas with isolated projects in separate repos I can let them run more freely, then review in gitkraken before committing.

Re: A Research Preview of Codex

#84

Post by flakiness »

yanis_t wrote: post #54

So it's looking like it's only running in the cloud, that is it will push commits to my remote repo before I have a chance to see if it works? When I'm using aider, after it make a commit what I do, I then immediately run git reset HEAD^ and then git diff (actually I use github desktop client to see the diff) to evaluate what exactly it did, and if I like it or not. Then I usually make some adjustments and only after…

You can think of this as a managed (cloud) version of their codex command line tool, which runs locally on your laptop.

The secret sauce here seems like their new model, but I expect it to come to API at some point.

Re: A Research Preview of Codex

#85

Post by tough »

enjoylife wrote: post #66
awestroke wrote: post #50

Earlier quoted context omitted.

At the end of the day, it's your job to deliver value. If a tool allows you to deliver more faster, without sacrificing quality, it's your responsibility to use that tool. You'll just have to make sure you can fully take responsibility for the end deliverables. And these tools are not only useful for writing the final code

> these tools are not only useful for writing the final code This sparked a thought in how a large part of the job is often the work needed to demonstrate impact. I think this aspect is often overlooked by some of the good engineers not yet taking advantage of the AI tooling. LLM loops may not yet be good enough to produce shippable code by themselves, but they sure are capable to help reduce the overhead of these up…

you mean like hacking a first POC with AI to sell a product/feature internally to get buy-in from the rest of the team before actually shipping production version of it?

Re: A Research Preview of Codex

#86

Post by arkmm »

prhn wrote: post #12

Is anyone using any of these tools to write non boilerplate code? I'm very interested. In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it. These tools aren't useless. They're great as search engines and pointing me i…

I think most code these days is boilerplate, though the composition of boilerplate snippets can become something unique and differentiated.

Re: A Research Preview of Codex

#87

Post by tough »

can someone give me a test prompt to one-shot something in go for testing?

(Im trying something)

what would be an impressive program that an agent should be able to one-shot in one go?

Re: A Research Preview of Codex

#88

Post by fabmilo »

tough wrote: post #20
tekacs wrote: post #14

Earlier quoted context omitted.

So -- that client-side thing is _technically_ called `codex-cli` (in the parent 'codex' repo, which looks like a monorepo?). Still super confusing, though! I feel like companies working with and shipping LLMs would do well to remember that it's not just humans who get confused by this, but LLMs themselves... it makes for a painful time, sending off a request and noting that a third of the way into its reasoning that…

they also have a dual implementation on rust and typescript there's codex-rs in that monorepo

more excited about the rust impl than the typescript one.

Re: A Research Preview of Codex

#89

Post by fourside »

johnjwang wrote: post #80

Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…

> You still need to do a lot of work to get it production ready, but it's as if you have an infinite number of junior engineers at your disposal now all working on different things.

One issue with junior devs is that because they’re not fully autonomous, you have to spend a non trivial amount of time guiding them and reviewing their code. Even if I had easy access to a lot of them, pretty quickly that overhead would become the bottleneck.

Did you think that managing a lot of these virtual devs could get overwhelming or are they pretty autonomous?

Re: A Research Preview of Codex

#90

Post by strangescript »

johnjwang wrote: post #80

Some engineers on my team at Assembled and I have been a part of the alpha test of Codex, and I'll say it's been quite impressive. We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas: Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run…

it feels like openai are at a ceiling with their models, codex1 seems to be another RLHF derivative from the same base model. You can see this in their own self reported o3-high comparison where at 8 tries they converge at the same accuracy.

It also seems very telling they have not mentioned o4-high benchmarks at all. o4-mini exists, so logically there is an o4 full model right?

Post reply on HN