Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Scientist: Towards Automated Open-Ended Scientific Discovery

sakana.ai

Post reply on HN
Print view View original

101–110 of 144 posts

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#101

Post by gessha »

defamation wrote: post #76
ai4ever wrote: post #72

Earlier quoted context omitted.

LLM have unleashed the dreamer in each and every young coder. Now, there is all sorts of speculation on what these machines can or cannot do. This is a natural process of any mania. These folks must all do courses in epistemology to realize that all knowledge is built up of symbolic components and not spit out by a probabilistic machine. Gradually, reality will sync (intentional misspelling) in, and such imaginations…

my guy you're so confident yet you forget AlphaFold, it designs protein structures that don't exist. Who's to say that a model can't eventually be trained to work within certain parameters the real word operates in and make new novel ideas and inventions much like a human does in a larger scope.

I’ll believe it when I see it and/or when I see the research path that goes there.

Judge a technology based on what it’s currently capable of and not what it promises to be.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#102

Post by Retric »

visarga wrote: post #80
JBorrow wrote: post #61

As someone 'in academia', I worry that tools like this fundamentally discard significant fractions of both the scientific process and why the process is structured that way. The reason that we do research is not simply so that we can produce papers and hence amass knowledge in an abstract sense. A huge part of the academic world is training and building up hands-on institutional knowledge within the population so tha…

> If I can just tab complete my N body code, did I really gain the knowledge of building it? Yes, because fixing it requires about the same effort as writing it from scratch. At least with current level of AI. When it works well, you just move it to a library and use it without worrying about implementation, like we do with all other library code. Using AI doesn't make the problem any easier for the developer. The fa…

> fixing it requires about the same effort as writing it from scratch

Today’s research AI doesn’t work, but that’s independent from why a working version would be problematic.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#103

Post by vouaobrasil »

whamlastxmas wrote: post #77
vouaobrasil wrote: post #47

As someone who truly loves science, the idea of automating the creative parts strikes me at the core as a horrible mistake. Yes, even before AI, we've already tried some automations -- actually some of those I even believe is a bad thing, such as the internet. Most people would disagree no doubt, but I feel like automating science, especially with regard to the more "creative parts" makes it more like an industry, ri…

I think this is looking at it wrong. If AI can do boring science it frees us up to do imaginative and fun science without the constraints of capitalism. You don’t have to worry about your science being valuable enough

> AI can do boring science it frees us up to do imaginative and fun science without the constraints of capitalism.

That is senseless. Capitalism will always control science by its very nature: through science, people create value and trade it for other things.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#104

Post by ben_w »

gwervc wrote: post #96
ben_w wrote: post #60

Earlier quoted context omitted.

> advocating this is a likely outcome of creating spam generators is laughable They're used as spam generators because they're cheap. The quality in many fields is currently comparable to someone in the middle of a degree in that field, which makes the quoted comparison a bit like the time Pierre Curie stuck a lump of radium on their arm for ten hours to see what it would do. I can imagine him reacting "What's that y…

> They're used as spam generators because they're cheap. No, the current fade of IA (LLM) are text generators. Very good, but nothing more than that. > there is no upper bounds to how much harm can be done by putting an idiot in charge of things Which is the not an AI problem. An AI may kill people indirectly in a setup like emergency services chatbot and a bad decision is taken, but it certainly couldn't roam the st…

> No, the current fade of IA (LLM) are text generators. Very good, but nothing more than that.

That doesn't address the point, and is also false.

Transformers are token generators, which means they can also do image and sound, and DNA sequences.

But even if they were just text, source code is "just text", laws are "just text", contract documents are "just text".

They have been used to control robots, both as input and output.

> Which is the not an AI problem

"Good news, at least 3,787 have died and it might be as bad as 16,000!"

"How is that good news?"

"We're an AI company, and it was our AI which designed and ran the pesticide plant that exploded in a direct duplication of everything that went wrong at Bohpal."

"Again, how is this good news?"

"We can blame the customer for using our product wrong, not our fault, yay!"

"I'm sure the victims and their family will be thrilled to learn this."

> it certainly couldn't roam the street with a kalachnikov killing people randomly or stabbing children

It can when it's put in charge of a robot body.

There's multiple companies demonstrating this already.

Pretending that AI can't be used to control robots is like saying that nothing that happens on the internet has any impact on real life.

Fortunately the AI which have been given control of robot bodies so far aren't doing that — want to risk your life with the humanoid robot equivalent of the Uber self driving car?

> The proponents of "AI can kill us all" can't write a single likely and non-contrived example of how that could happen.

Anything less would be a thing we can trivially prevent.

It's not like "dig up all the fossil fuels and burn them despite public protest about climate change and the existence of alternatives, and suing the protesters with SLAPP suits so we can keep doing it because it's inconvenient to believe the science and even if it did the consequences won't affect us personally", doesn't sound contrived.

And that's with humans making the decisions, humans whose grandkids would be affected.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#105

Post by jltsiren »

Rebuff5007 wrote: post #93
OutOfHere wrote: post #15

To produce scientific work, one needs certain raw materials: 1. Data 2. Access to past works Once you have these, only then can discoveries can be made, and papers be written. How does this software get these? I am assuming they have to be provided up-front to the software for each job.

To produce scientific work, one needs to follow the scientific method. This involves stating a hypothesis, designing an experiment that would test this hypothesis, conducting the experiment with controls, and analyzing the data w.r.t to the hypothesis being tested. Access to past works is only useful in informing what is a good hypothesis worth testing. And data is only useful when generated by an experiment that is…

The kind of cargo cult science you describe is the main reason for the replication crisis. The more people believe that they will reach true knowledge by following the sacred rituals to the letter, the more likely they will do things in the established ways without thinking and repeat the same mistakes over and over again.

In actual science, the key step is stopping to question yourself all the time. Does the thing you were planning to do still make sense? Especially in the current context? Given what you have seen so far? Should you change your plans? If so, can the work you have already done be salvaged? Or do you have to discard it and start over from the beginning?

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#106

Post by plaidfuji »

When “executing the experiment” amounts to modifying ~50 lines of PyTorch code tweaking model architecture, I’d bloody well expect that you can automate it.

That’s not “automating scientific discovery”, that’s “procedurally optimizing model architecture” (and one iteration of exploration at that!). In any other field of science the actual work and data generated by the AI Scientist would be a sub-section of the Supporting Info if not just a weekly update to your advisor.

Don’t get me wrong, the actual work done by the humans who are publishing this is a pretty solid piece of engineering and interesting to discuss. But the automated papers, to me, are more a commentary on what constitutes a publishable advancement in AI these days.

Edit: this also further confirms my suspicion about LLMs, which is that they aren’t very good at doing actual work, but they are great at generating the accompanying marketing BS around having done work. They will generate a mountain of flashy but frivolous communication about smaller and smaller chunks of true progress, which while appearing beneficial to individuals, will ultimately result in a race to the bottom of true productivity.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#107

Post by tossandthrow »

JBorrow wrote: post #61

As someone 'in academia', I worry that tools like this fundamentally discard significant fractions of both the scientific process and why the process is structured that way. The reason that we do research is not simply so that we can produce papers and hence amass knowledge in an abstract sense. A huge part of the academic world is training and building up hands-on institutional knowledge within the population so tha…

You don't need to understand electromagnetism in order to watch television.

Also, institutions without a purpose should not be kept going.

Ie. Ai needs to take over the entire life cycle for research before this is real issues. I don't see that happening anytime soon.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#108

Post by tzumaoli »

woah wrote: post #95

Everyone in this thread is musing about the role of AI and whether the process of discovery is fundamentally human, and what Isaac Newton would think, but can somebody tell me: is the technology it develops any good? For example, does "Dual Scale Diffusion" https://sakana.ai/assets/ai-scientist/adaptive_dual_scale_de... look useful?

Discussed in another thread https://news.ycombinator.com/item?id=41234415 As someone who has worked on diffusion model, it's a clear reject and not a very interesting architecture. The idea is to train a diffusion model to fit to low dimensional data using two MLPs: one accounts for high-level structure and one accounts for low level details. These kind of "global-local" architecture is very common in computer vision/graphics (with the paper mentioned none of the relevant work), so the novelty is low. The experiments also do not clearly showcase where exactly this "dual" structure brings benefits.

That being said, it's very hard to tell it apart from a normal poorly-written paper from a quick glance. If you tell me it's written by a graduate student, I would probably believe it. It is also interesting in a way that maybe for low-dimensional signals there are some architecture tweaks we can do to modify the existing diffusion model architectures to make things better, so maybe not 100% BS.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#109

Post by ben_w »

exe34 wrote: post #88
DennisP wrote: post #52

Earlier quoted context omitted.

It's possible for current harms and future risks to both be real. It's also possible for human civilization to address more than one problem at a time. "You care about X but that's just a distraction from the thing I care about which is Y" is not really a good argument. I could just as well say that copyright concerns are just a distraction from the risk that AI could kill us all. And it seems to me that if the AI in…

> And it seems to me that if the AI industry wanted to distract us from harms, they would give us optimistic scenarios. Nah it has to appear plausible.

People are very good at promising a better future in a non-specific way and without much evidence. That's kinda how Brexit happened.

It's when you get the specific details of a utopia that you upset people — for example, every time I see anti-aging discussed here, there's a bunch of people for whom that is a horror story. I can't imagine being them, and they can't imagine being me.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#110

Post by anticensor » 

  Presentation of Intermediate Results. The paper contains results
  for every single experiment that was run. While this is useful
  and insightful for us to see the evolution of the idea during
  execution, it is unusual for standard papers to present
  intermediate results like this.
This is actually quite good that the AI scientist does this. AI has no excuse of slow report writing that humans have to omit the intermediate results.
Post reply on HN