Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Scientist: Towards Automated Open-Ended Scientific Discovery

sakana.ai

Post reply on HN
Print view View original

11–20 of 144 posts

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#11

Post by viraptor »

It's interesting that the cost total is the same in all tables. I can't tell if that's a copy&paste error, or the cost was capped, or are the totals for all the experiments?

> Aider fails to implement a significant fraction of the proposed ideas.

Yeah, that can be improved a lot with a better agent for code. While aider is fast and cheap, going with something like plandex or opendavin makes a massive difference... both in quality and cost. For example plandex will burn $1 on a simple script, but I can expect that script to work as requested. A mixed approach could be deepseek coder with an agent - a bit worse quality, but still cheaper to do more iterations.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#13

Post by mbladra »

>The AI Scientist automates the entire research lifecycle, from generating novel research ideas, writing any necessary code, and executing experiments, to summarizing experimental results, visualizing them, and presenting its findings in a full scientific manuscript.

I'd be curious how much of the experimentation process companies like OAI/Anthropic have automated by now, for improving their own models.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#15

Post by OutOfHere »

To produce scientific work, one needs certain raw materials:

1. Data

2. Access to past works

Once you have these, only then can discoveries can be made, and papers be written. How does this software get these? I am assuming they have to be provided up-front to the software for each job.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#17

Post by visarga »

OutOfHere wrote: post #15

To produce scientific work, one needs certain raw materials: 1. Data 2. Access to past works Once you have these, only then can discoveries can be made, and papers be written. How does this software get these? I am assuming they have to be provided up-front to the software for each job.

You are right but didn't go deep enough: you need interactive data. Not just static data. Environments.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#18

Post by llm_trw »

mbladra wrote: post #16
ks2048 wrote: post #12

So, I assume journals will need AIs to do scientific review to handle the flood of AI-produced paper submissions?

Or just keyword search new submissions for the word "delve"

Just add "don't use delve" into the system prompt.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#19

Post by csto12 »

mbladra wrote: post #13

>The AI Scientist automates the entire research lifecycle, from generating novel research ideas, writing any necessary code, and executing experiments, to summarizing experimental results, visualizing them, and presenting its findings in a full scientific manuscript. I'd be curious how much of the experimentation process companies like OAI/Anthropic have automated by now, for improving their own models.

I’m wondering the same. If for example, to create GPT-5, OpenAI could credit some % of progress due to research conducted by LLMs with minimal/no human interaction, it would be a nice marketing piece.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#20

Post by a_bonobo »

minihat wrote: post #9

Clarkesworld sci-fi magazine temporarily closed submissions due to low quality AI spam. I'm sure the irony will not be lost on them if ML journals are the next victims.

For AI journals it might even be the opposite: researchers running these tools pay OA fees for the submission, the journals make bank, the universities rise in research rankings due to more papers published, everybody is happy. Who needs real progress if the paper number goes up?
Post reply on HN