Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Scientist: Towards Automated Open-Ended Scientific Discovery

sakana.ai

Post reply on HN
Print view View original

91–100 of 144 posts

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#91

Post by the__alchemist »

defamation wrote: post #76
ai4ever wrote: post #72

Earlier quoted context omitted.

LLM have unleashed the dreamer in each and every young coder. Now, there is all sorts of speculation on what these machines can or cannot do. This is a natural process of any mania. These folks must all do courses in epistemology to realize that all knowledge is built up of symbolic components and not spit out by a probabilistic machine. Gradually, reality will sync (intentional misspelling) in, and such imaginations…

my guy you're so confident yet you forget AlphaFold, it designs protein structures that don't exist. Who's to say that a model can't eventually be trained to work within certain parameters the real word operates in and make new novel ideas and inventions much like a human does in a larger scope.

AlphaFold doesn't solve the protein folding problem. It has practical applications, but IMO we still need to (And can!) build better ab-initio chemistry models that will actually simulate protein folding, or chemical reactions more generally.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#92

Post by EvgeniyZh »

EGreg wrote: post #57
foolfoolz wrote: post #32

Earlier quoted context omitted.

we already have a wave of papers that no human has the capacity to verify

Literally every AI discussion on HN has the same format > AI is awesome, regulation is stupid > I wouldnt want to see AI flooding the market with X we can’t verify > We already have (copy whatever was just said) For what it’s worth, when it comes to SCIENCE, I an actually in favor of AI, even giving it to everyone. Except possibly AI that would help engineer designer viruses. Because in science, people literally ARE…

> If AI churns out 25 scientific models per day, the proper venue isnt publishing them on arxiv or Nature magazine. It’s testing their predictions and putting them up the same way HuggingFace does.

Unless the testing will be done by AI, I doubt human scientist will bother with testing tons of incomprehensible models, unless the accuracy of these models will be exceptionally high

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#93

Post by Rebuff5007 »

OutOfHere wrote: post #15

To produce scientific work, one needs certain raw materials: 1. Data 2. Access to past works Once you have these, only then can discoveries can be made, and papers be written. How does this software get these? I am assuming they have to be provided up-front to the software for each job.

To produce scientific work, one needs to follow the scientific method. This involves stating a hypothesis, designing an experiment that would test this hypothesis, conducting the experiment with controls, and analyzing the data w.r.t to the hypothesis being tested.

Access to past works is only useful in informing what is a good hypothesis worth testing. And data is only useful when generated by an experiment that is testing for causality (see [1]).

[1] https://pyimagesearch.com/2023/11/27/a-brief-introduction-to...

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#94

Post by nprateem »

ineedaj0b wrote: post #28
unraveller wrote: post #7

But OpenAI said LLMs can't innovate until human-level reasoning and long-term agenthood is solved. [1] Referring to their precious 5 stages to classify AI before it reaches the scary "beyond" levels of intelligence... presumably at that point they get the feds involved to reg cap the field, so genuine is the fear of the pace they've set. It's clear OpenAI is a hype company knocking over glass bottle stacks at its own…

sam made comments on Twitter we hit level 2 - we’ll know more in the coming weeks if he’s right.

Highly unlikely since today's models can't consistently follow simple instructions.

Eg "don't waffle, don't sound like you're writing an essay, don't use the fucking word delve, don't apologise"

"Sorry about that, let's delve into this"

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#95

Post by woah »

Everyone in this thread is musing about the role of AI and whether the process of discovery is fundamentally human, and what Isaac Newton would think, but can somebody tell me: is the technology it develops any good? For example, does "Dual Scale Diffusion" https://sakana.ai/assets/ai-scientist/adaptive_dual_scale_de... look useful?

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#96

Post by gwervc »

ben_w wrote: post #60
gwervc wrote: post #50

Earlier quoted context omitted.

> This may kill literally everyone It's indeed hard to take seriously such gross exaggeration. Even the deadliest plagues didn't kill everyone, so advocating this is a likely outcome of creating spam generators is laughable. This is more likely a strategy, common in academia, of aggrandizing results (here risks) so that more eyeballs, attention and money is diverted towards the field and its proponents.

> advocating this is a likely outcome of creating spam generators is laughable They're used as spam generators because they're cheap. The quality in many fields is currently comparable to someone in the middle of a degree in that field, which makes the quoted comparison a bit like the time Pierre Curie stuck a lump of radium on their arm for ten hours to see what it would do. I can imagine him reacting "What's that y…

> They're used as spam generators because they're cheap.

No, the current fade of IA (LLM) are text generators. Very good, but nothing more than that.

> there is no upper bounds to how much harm can be done by putting an idiot in charge of things

Which is the not an AI problem. An AI may kill people indirectly in a setup like emergency services chatbot and a bad decision is taken, but it certainly couldn't roam the street with a kalachnikov killing people randomly or stabbing children (and if that ever happens politicians will say this has nothing to do with AI). The proponents of "AI can kill us all" can't write a single likely and non-contrived example of how that could happen.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#97

Post by happypumpkin »

Potential concerns with their self-eval:

They evaluate their automated reviewer by comparing against human evaluations on human-written research papers, and then seem to extrapolate that their automated reviewer would align with human reviewers on AI-written research papers. It seems like there are a few major pitfalls with this.

First, if their systems aren't multimodal, and their figures are lower-quality than human-created figures (which they explicitly list as a limitation), the automated reviewer would be biased in favor of AI-generated papers (only having access to the text). This is an obvious one but I think there could easily be other aspects of papers where the AI and human reviewers align on human-written papers, but not on AI papers.

Additionally, they note:

> Furthermore, the False Negative Rate (FNR) is much lower than the human baseline (0.39 vs. 0.52). Hence, the LLM-based review agent rejects fewer high-quality papers. The False Positive Rate (FNR [sic]), on the other hand, is higher (0.31 vs. 0.17)

It seems like false positive rate is the more important metric here. If a paper is truly high-quality, it is likely to have success w/ a rebuttal, or in getting acceptance at another conference. On the other hand, if this system leads to more low-quality submissions or acceptances via a high FPR, we're going to have more AI slop and increased load on human reviewers.

I admit I didn't thoroughly read all 185 pages, maybe these concerns are misplaced.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#98

Post by happypumpkin »

happypumpkin wrote: post #97

Potential concerns with their self-eval: They evaluate their automated reviewer by comparing against human evaluations on human-written research papers, and then seem to extrapolate that their automated reviewer would align with human reviewers on AI-written research papers. It seems like there are a few major pitfalls with this. First, if their systems aren't multimodal, and their figures are lower-quality than huma…

Also a concern about the paper generation process itself:

> In a similar vein to idea generation, The AI Scientist is allowed 20 rounds to poll the Semantic Scholar API looking for the most relevant sources to compare and contrast the near-completed paper against for the related work section. This process also allows The AI Scientist to select any papers it would like to discuss and additionally fill in any citations that are missing from other sections of the paper.

So... they don't look for related work until the paper is "near-completed." Seems a bit backwards to me.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#99

Post by _heimdall »

sebastiennight wrote: post #87
_heimdall wrote: post #75

Earlier quoted context omitted.

That sounds like an extremely dangerous day for science as well. If anyone could pop up an ML tool and task it with inventing and validating something truly novel, that would be weaponized extremely fast (likely right after people use it for porn, the frontier for all new tech). I do totally agree on the cost + margins point you make. I've never actually been a fan of valuing things in that way, and in my pipe dream…

I would compare this question to "creating a new page on the Internet just adds to a countless pile of URLs. How important can any one really be?" And this leads us to: most will be slop, but if you can figure out effective ways to perform (a) Search, and (b) Alerts, then this scenario is definitely a game-changer. Let's take protein synthesis: imagine if we were able to programmatically generate an accurate paper de…

That actually raises a more fundamental question here.

This project specifically focused their tests on research topics that can feasibly be tested by the ML tools, writing software. I assume that was an intentional decision, and a clever one that let them point to promising test results while ignoring that potential limitation when valuing their tool.

These ML tools will need to not only come up with novel ideas, they'll need a way to test and validate them. For anything outside of software that almost certainly means modelling. If we already have validated models that may work well enough, but if you extend the scope to literally any novel protein that is possible the ML tool would first have to figure out how to model it.

What would that even look like? How would an ML tool trapped in a computer and limited to the knowledge it was trained on be able to model any protein in the universe and be able to validate exactly how it would function in the real world?

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#100

Post by jltsiren »

pmayrgundter wrote: post #43
zipy124 wrote: post #40

As a scientist in academic research, I can only see this as a bad thing. The #1 valued thing in science is trust. At the end of the day (until things change in how we handle research data, code etc...) all papers are based on the reviewers trust in the authors that their data is what they say it is, and the code they submit does what it says it does. Allowing an AI agent to automate code, data or analysis, necessitat…

Maybe the #1 valued thing in "capital S Science" -- the institutional bureaucracy of academia -- is trust. Trust that the bureaucracy will be preserved, funded, defended.. so long as the dogma is followed. The politics of Science. The #1 valued thing in science is the method of doing science: reason, insight, objectivity, evidence, reproducibility. If the method can be automated, then great!

Trust is the primary value, because it covers everything you listed.

Most people who read research papers only skim through the paper to get the big picture. They trust that the authors and the publication system did a good-faith effort to advance science. If they can't trust that, they almost certainly won't read the paper, because they don't have the time and the interest to go through the details. Only a few people read the technical parts with the intent to understand them. Even fewer go through supplementary materials and external reproducibility instructions with a similar attention to detail.

Post reply on HN