GPT-5-Codex is a better AI researcher than me
seangoedecke.com
GPT-5-Codex is a better AI researcher than me
1–10 of 39 posts
Re: GPT-5-Codex is a better AI researcher than me
#2It's like reading a self-written obituary of someone who stopped thinking for themselves.
They aren't excited about anything. They aren't in awe. They haven't done any hard work. They're just here to ooze lukewarm sludge
Re: GPT-5-Codex is a better AI researcher than me
#3I love using AI to set up projects in 5 minutes but I hate to develop these projects using AI because inevitably it runs into a wall and I need to guide it and fix its code.
I suppose in this case it picked up an existing project and DIDN’T walk off a cliff? Were the mutations really small?
Re: GPT-5-Codex is a better AI researcher than me
#4The title makes it sound like we have reached the singularity. The real insight here is that amateurs may have a difficult time competing with AI
Re: GPT-5-Codex is a better AI researcher than me
#5I’ve been putting questions into LLM research functions, including Claude’s research mode, and letting them churn until a report appears.
I’ve been starting with topics where I’m already familiar with the answer but want a refreshed. So far, I’m not impressed. Some times the info will be correct. Most of the time it strings together a lot of words from the material it finds but it reads like an undergrad trying to paraphrase the Wikipedia page without understanding the content. Often it will have one bullet point that is completely wrong.
The other problem I’m having is that it’s not very good at identifying poor sources. This is less of a problem with topics like math and engineering, but a big problem with topics like health and medicine where it will pick up alternative medicine and pseudoscience pages and integrate them into the research as if they were real. There are a lot of health and medicine topics where the way pseudoscience people talk about a subject doesn’t match the real science, but they use the same words and therefore catch the same search terms.
An example is the way “dopamine” is used in casual conversation and by influencers in ways that aren’t accurate. Concepts like “dopamine fasting” or claiming things “raise your dopamine” aren’t scientifically accurate but use the same words nevertheless and therefore can get pulled into the training set and searches.
Re: GPT-5-Codex is a better AI researcher than me
#6Ah, another weekly AI booster article from the AI booster gang.
Hate to break it to you, but if GPT-5 is better AI researcher than you, you were probably not that good to begin with.
Does Codex tell you why 95% of AI projects in the enterprise fail? Or why the only study up to date on merits of AI for coding shows 19% decrease of productivity.
Re: GPT-5-Codex is a better AI researcher than me
#7The author wasn't doing "AI research" before and neither was GPT5. This is not at the frontier of anything, it is just an already solved problem in training that GPT5 found. Had the author been willing to actually do a Google and GitHub search, or just twiddle the training knob parameters enough on their own, they would have found a better solution than working alone.
Also this footnote:
> Alone” here is relative - I did use ChatGPT and a bit of Copilot to generate some of the training code in my last attempt. I just didn’t use any agentic tool
I have no words. I wonder if this "AI researcher" can make it through the original Attention Is All You Need paper without an LLM.
Re: GPT-5-Codex is a better AI researcher than me
#8So why is "distilling from N-gram" better, why does it make the transformer learn English faster?
Hypothesis: it's the standard "teacher-student" or "distillation" trick - if you're learning next-token-prediction, you only learn what the correct answer is (i.e. the spike in probability), but when you're distilling from a teacher model, you learn the entire distribution of potential answers.
Curious, can anyone more experienced in AI research comment on this?
Re: GPT-5-Codex is a better AI researcher than me
#9I’ve been putting questions into LLM research functions, including Claude’s research mode, and letting them churn until a report appears. I’ve been starting with topics where I’m already familiar with the answer but want a refreshed. So far, I’m not impressed. Some times the info will be correct. Most of the time it strings together a lot of words from the material it finds but it reads like an undergrad trying to pa…
Claude's research mode is by far the worst one I've used, I consider it nearly useless. I cannot trust it specifically because Anthropic has a policy of refusing to use reddit for anything, whether it be as a research source or in claude chrome.
Reddit may not be the greatest source for hard science, but for things like "tell me what shoes people are finding helpful for their plantar fasciitis" I appreciate reddit's anecdata of reddit over every other source.
Re: GPT-5-Codex is a better AI researcher than me
#10> OpenAI has released GPT-5-codex, and supposedly uses it ... to automate a lot of their ... AI research
If I was the owner of an AI company that was forever trying to juice its valuation and raise money, you can bet I'd be telling people I had built a magic self-improving AI.