Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Scientist: Towards Automated Open-Ended Scientific Discovery

sakana.ai

Post reply on HN
Print view View original

51–60 of 144 posts

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#51

Post by hoseja »

VieEnCode wrote: post #44
ben_w wrote: post #36

Earlier quoted context omitted.

> It's clear OpenAI is a hype company Every other industry: "My new invention is safe, I swear" Public reaction: "You're biased, it's dangerous!" Almost the entire AI industry, including people who resign to speak more openly about the risks: "This may kill literally everyone, none of us knows what we're doing or what 'safe' even means" Public reaction: "You're biased, it's safe, just saying that to look cool!"

I found this guy's take on the AI safety scene to be quite insightful. In summary, he feels the focus on sci-fi type existential risk to be a deliberate distraction from the AI industry's current and real legal and ethical harms: e.g. scraping copyrighted content for training without paying or attributing creators, not protecting those affected by the misuse of tools to create deepfake porn, the crashes and deaths at…

Only the last one is in any way actually bad and even then it should be in the interest of the company using it to fix it promptly.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#52

Post by DennisP »

VieEnCode wrote: post #44
ben_w wrote: post #36

Earlier quoted context omitted.

> It's clear OpenAI is a hype company Every other industry: "My new invention is safe, I swear" Public reaction: "You're biased, it's dangerous!" Almost the entire AI industry, including people who resign to speak more openly about the risks: "This may kill literally everyone, none of us knows what we're doing or what 'safe' even means" Public reaction: "You're biased, it's safe, just saying that to look cool!"

I found this guy's take on the AI safety scene to be quite insightful. In summary, he feels the focus on sci-fi type existential risk to be a deliberate distraction from the AI industry's current and real legal and ethical harms: e.g. scraping copyrighted content for training without paying or attributing creators, not protecting those affected by the misuse of tools to create deepfake porn, the crashes and deaths at…

It's possible for current harms and future risks to both be real. It's also possible for human civilization to address more than one problem at a time. "You care about X but that's just a distraction from the thing I care about which is Y" is not really a good argument. I could just as well say that copyright concerns are just a distraction from the risk that AI could kill us all.

And it seems to me that if the AI industry wanted to distract us from harms, they would give us optimistic scenarios. "Sure these are problems but it will be worth it because AI will give us utopia." That would be an argument for pushing forward with AI.

Instead we're getting "oh, you may think we have problems now but that's nothing, a few years from now it's going to kill us all." Um, ok, I guess full steam ahead then? If this is a marketing campaign, it's the worst one in history.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#53

Post by frotaur »

shusaku wrote: post #27

Some samples of the generated papers are in the SI of their paper. It’s be interesting if some of you ML guys dug into them. The fact that they built another AI system to review the papers seems really shaky, this is where human feedback would be most valuable.

Tried reading the 'low-dimensional diffusion' one. Not an expert on diffusion by any means, but the very premise of the paper seems like bullshit.

It claims that 'while diffusion works in high-dimensional datasets, it struggle in low-dimensional settings', which just makes no sense to me? Modeling high-dimensional data is just strictly harder than low-dimensional one.

Then when you read the intro, it's full of 'blanket statements' about diffusion, which have nothing to do with the subject, e.g. 'The challenge in applying diffusion models to low-dimensional spaces lies in simultaneously capturing both the global structure and local details of the data distribution. In these spaces, each dimension carries significant information about the overall structure, making the balance between global coherence and local nuance particularly crucial.'

I really don't see the connection between global structure/local details and low-dimensional data.

The graphs also make no sense. Figure 1 is just almost the same graph repeated 6 times, for no good reason.

It uses an MLP as its diffusion model, which is kinda ridiculous compared to what's the now-established architectures (U-net/ vision transformer based models). Also, the data it learns on is 2-dimensional. I get that the point is using low-dimensional data, but there is no way that people ever struggle with it. Case in point, they solve it with 2-layer MLPs, and it has probably nothing to do with their 'novel multi-scale noise' (since they haven't compared to the 'non-multiscale' version).

Finally, it cites mostly only each field's 'standard' papers, doesn't cite anything really relevant to what it does.

Overall, it looks exactly like what you would expect out of GPT-generated paper, just reshashing some standard stuff in a mostly coherent piece of garbage. I really hope people don't start submitting this kind of stuff to conferences.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#54

Post by DennisP »

gwervc wrote: post #50
ben_w wrote: post #36

Earlier quoted context omitted.

> It's clear OpenAI is a hype company Every other industry: "My new invention is safe, I swear" Public reaction: "You're biased, it's dangerous!" Almost the entire AI industry, including people who resign to speak more openly about the risks: "This may kill literally everyone, none of us knows what we're doing or what 'safe' even means" Public reaction: "You're biased, it's safe, just saying that to look cool!"

> This may kill literally everyone It's indeed hard to take seriously such gross exaggeration. Even the deadliest plagues didn't kill everyone, so advocating this is a likely outcome of creating spam generators is laughable. This is more likely a strategy, common in academia, of aggrandizing results (here risks) so that more eyeballs, attention and money is diverted towards the field and its proponents.

It's quite common for new species to kill off old species. We ourselves have obliterated many species that we outcompeted for resources.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#55

Post by eltoxo »

DennisP wrote: post #54
gwervc wrote: post #50

Earlier quoted context omitted.

> This may kill literally everyone It's indeed hard to take seriously such gross exaggeration. Even the deadliest plagues didn't kill everyone, so advocating this is a likely outcome of creating spam generators is laughable. This is more likely a strategy, common in academia, of aggrandizing results (here risks) so that more eyeballs, attention and money is diverted towards the field and its proponents.

It's quite common for new species to kill off old species. We ourselves have obliterated many species that we outcompeted for resources.

As if software is the same thing as a new biological species.

I am just so bored of reading bullshit like this.

If you really believe this then you need to level up your level of education and learning. It is not good.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#56

Post by roenxi »

gwervc wrote: post #50
ben_w wrote: post #36

Earlier quoted context omitted.

> It's clear OpenAI is a hype company Every other industry: "My new invention is safe, I swear" Public reaction: "You're biased, it's dangerous!" Almost the entire AI industry, including people who resign to speak more openly about the risks: "This may kill literally everyone, none of us knows what we're doing or what 'safe' even means" Public reaction: "You're biased, it's safe, just saying that to look cool!"

> This may kill literally everyone It's indeed hard to take seriously such gross exaggeration. Even the deadliest plagues didn't kill everyone, so advocating this is a likely outcome of creating spam generators is laughable. This is more likely a strategy, common in academia, of aggrandizing results (here risks) so that more eyeballs, attention and money is diverted towards the field and its proponents.

> Even the deadliest plagues didn't kill everyone...

That is logically flawed; the species that were killed off by plagues aren't around to say that. Every species exists in a state of "the deadliest plagues [we've experienced so far] didn't kill everyone". You can say that about literally every threat - we know we have overcome everything thrown at us so far because we are still here. That will continue to be the case for everything that humanity ever faces except for 1 thing (we aren't certain what yet).

But we know that species go extinct from time to time, so the logic that we've overcome things in the past ergo we are safe doesn't make sense for ruling out even many well known threats. Let alone systems that can outplan us; we've never faced inhuman competitors that can strategise more effectively than a human.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#57

Post by EGreg »

foolfoolz wrote: post #32
uniqueuid wrote: post #25

Earlier quoted context omitted.

It would also be sad to see the scientific system destroyed by a wave of automatically generated papers that no human has the capacity to verify. It's not hard to generate ideas, it's hard to generate reliable and relevant ideas. Such AI science generators are destroying the grass they graze on unless they take science more seriously (and not as a toddler idea of "generating and testing ideas", which is only a small…

we already have a wave of papers that no human has the capacity to verify

Literally every AI discussion on HN has the same format

  > AI is awesome,
    regulation is stupid

    > I wouldnt want to see
      AI flooding the market
      with X we can’t verify
      
      > We already have (copy
        whatever was just said)
For what it’s worth, when it comes to SCIENCE, I an actually in favor of AI, even giving it to everyone. Except possibly AI that would help engineer designer viruses.

Because in science, people literally ARE just doing pattern matching and testing the patterns. Kepler just looked at a bunch of star charts and found a low-dimensional explanation with ellipses. You can throw that data at an AI and it will come up with physics laws involving 24,729 variables that predict things way more accurately. Including potentially chaotic systems like weather or a three body problem etc.

So yeah, use AI for that, because you can actually check its predictions against data, and develop a reputation. We can’t really reason about theories and models of systems with 30,000 variables anyway.

If AI churns out 25 scientific models per day, the proper venue isnt publishing them on arxiv or Nature magazine. It’s testing their predictions and putting them up the same way HuggingFace does. To amass reputation and be checked against real data by multiple parties.

As a side question — is protein folding @ home dead now, since Google “solved it” with massive clusters and AI?

What about SETI@home? Why not just run AI for SETI and solve the drake equation once and for all? :)

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#58

Post by hifromwork »

hoseja wrote: post #51
VieEnCode wrote: post #44

Earlier quoted context omitted.

I found this guy's take on the AI safety scene to be quite insightful. In summary, he feels the focus on sci-fi type existential risk to be a deliberate distraction from the AI industry's current and real legal and ethical harms: e.g. scraping copyrighted content for training without paying or attributing creators, not protecting those affected by the misuse of tools to create deepfake porn, the crashes and deaths at…

Only the last one is in any way actually bad and even then it should be in the interest of the company using it to fix it promptly.

Deaths in car crashes and copyright laundering by big corporations are not bad in any way at all?

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#59

Post by ben_w »

eltoxo wrote: post #55
DennisP wrote: post #54

Earlier quoted context omitted.

It's quite common for new species to kill off old species. We ourselves have obliterated many species that we outcompeted for resources.

As if software is the same thing as a new biological species. I am just so bored of reading bullshit like this. If you really believe this then you need to level up your level of education and learning. It is not good.

> As if software is the same thing as a new biological species.

The other poster didn't claim they were.

They don't need to be.

They don't even need to be given control of robotic bodies, though they already are.

What they do need to be, is competing for the same resources.

And there's plenty of examples of corporations doing things that are bad for humans in the long-term because they are good for short-term shareholder value. And filing SLAPP suits against any activist trying to stop them.

Re: The AI Scientist: Towards Automated Open-Ended Scientific Discovery

#60

Post by ben_w »

gwervc wrote: post #50
ben_w wrote: post #36

Earlier quoted context omitted.

> It's clear OpenAI is a hype company Every other industry: "My new invention is safe, I swear" Public reaction: "You're biased, it's dangerous!" Almost the entire AI industry, including people who resign to speak more openly about the risks: "This may kill literally everyone, none of us knows what we're doing or what 'safe' even means" Public reaction: "You're biased, it's safe, just saying that to look cool!"

> This may kill literally everyone It's indeed hard to take seriously such gross exaggeration. Even the deadliest plagues didn't kill everyone, so advocating this is a likely outcome of creating spam generators is laughable. This is more likely a strategy, common in academia, of aggrandizing results (here risks) so that more eyeballs, attention and money is diverted towards the field and its proponents.

> advocating this is a likely outcome of creating spam generators is laughable

They're used as spam generators because they're cheap.

The quality in many fields is currently comparable to someone in the middle of a degree in that field, which makes the quoted comparison a bit like the time Pierre Curie stuck a lump of radium on their arm for ten hours to see what it would do. I can imagine him reacting "What's that you say? A small lump of rock in a test tube might give me aplastic anemia*? The idea is laughtable!", except he'd probably have said that in Polish or French.

Even the limits of current models, even if we are using those models to their greatest potential (we're probably not), isn't a safety guarantee: there is no upper bounds to how much harm can be done by putting an idiot in charge of things, and the Peter Principle applies to AI as well as humans, as we're already seeing AI being used for tasks they are inadequate to perform.

* he died from a horse drawn cart, Marie Curie developed aplastic anemia and he likely would have too if not for the other accident getting him first.

Bonus irony: the general idea he had in regards to this, to use radiation to treat cancer, is correct and currently in use. They just didn't know anything like enough to do that at the time.

Post reply on HN