Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

651–660 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#651

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

This is the first time I've actually seen game theory applied to an actual real life scenario rather than just a theoretical academic problem. What's the best resource to learn more game theory and to be able to applying it like you to a more general setting?

Re: Pause Giant AI Experiments: An Open Letter

#652
“Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?”

What a bunch of nonsense, irrelevant questions. We’re nowhere near achieving any of these. Yann LeCun is one of the few voices of reason among so-called AI experts and he’s very clear about the capabilities and limitations of these models, and that these risks are currently irrelevant.

It took several people to carry the Optimus robot to the scene so it could spread its legs open, and Mr Musk is worried about AI taking all jobs in the near future?

Wonder if this is just ignorance or malice. This nonsense hype train overshadows the fascinating progress that’s actually been made. Let’s appreciate the actual progress and do away with the nonsense speculation.

Re: Pause Giant AI Experiments: An Open Letter

#653
post #563

Earlier quoted context omitted.

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering…

My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous. For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly th…

By that definition of AGI, not even most scientists are generally intelligent.

Re: Pause Giant AI Experiments: An Open Letter

#654

Earlier quoted context omitted.

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

I do viral bioinformatics for my job. Bioinformatics workflows analyze raw data to assemble sequences, create phylogenetic trees, etc. They can't just design a completely novel RNA sequence (this is not the same as de novo assembly). Scientists can definitely manipulate pre-existing genomes, synthesize the edited genome, and thereby synthesize viruses, but this involves a lot of trial-and-error, tedious wet lab work.…

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#655

Earlier quoted context omitted.

How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it. And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust O…

This has been repeated often, but even if it's true, I have to wonder why it's treated as a given with no further exploration. Is it because we as a species will inevitably accept any technological progress once sometime after it's been discovered, before the consequences can be suffered? What will that imply for any other species intelligent enough to get to where we are? The kinds of theories I mull over tend to de…

This is all speculative, but not fiction at this point clearly. Sci-fi authors have explored this possibility for decades, maybe their ideas could be of some help? I struggle to see how though; how to train Asimovs three laws for example?

Re: Pause Giant AI Experiments: An Open Letter

#656

“Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?” What a bunch of nonsense, irrelevant questions. We’re nowhere near achieving any of these. Yann LeCun is one of the few voices of reason among so-called AI experts and he’s very clear about the ca…

It all serves to inflate the self-importance of the bigwigs in this sector.

If you want adulation for your brilliance, go figure out what we need to go beyond the tired "scale transformers to infinity = superintelligence" meme that so many are stuck on.

Re: Pause Giant AI Experiments: An Open Letter

#657

“Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?” What a bunch of nonsense, irrelevant questions. We’re nowhere near achieving any of these. Yann LeCun is one of the few voices of reason among so-called AI experts and he’s very clear about the ca…

How long it takes for human child to start walking?

"it is too early, oops, no, it is too late".

Re: Pause Giant AI Experiments: An Open Letter

#658

Earlier quoted context omitted.

And add in just one peer-level war where one side has their back against the wall. Then give it 100 years where anyone can create such a model on their phone. We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens. To get this to work we need a far smarter entity with no p…

> In my darker thoughts about this, this is why we see no aliens. If AI would be a common great filter we'd expect at least one of them to expand outwards after being the filter?

Black holes == Super intelligence

(aka The Transcension Hypothesis)

https://www.sciencedirect.com/science/article/abs/pii/S00945...

Re: Pause Giant AI Experiments: An Open Letter

#659
This is the same sort of weak argument that crippled stem cell research. We could have been a decade ahead towards curing aging and innumerable ailments, but one loud group got to decide what's "ethical".

Will China or other countries stop their AI research? This is fundamentally a race for survival if you believe that AGI is an end-game. If one side of a future conflict is fielding robotic troops and AI controlled battalions, you'd better hope it's your side.

Stopping now would be like pausing the Manhattan project for ethical reasons. Nothing would have been achieved except delaying America, costing lives, and allowing the USSR to potentially have unilateral power.

If instead, current AI is just a "gloried auto-complete", then that makes this letter hilarious and equally pointless.

I feel like you can't have it both ways. If it's critical technology that will change the face of the future, it's important to get there while simultaneously taking the time to understand it (ala nuclear power). If it's not so critical, then we don't need a moratorium.

Edit: I think a better solution would be to have all the signatories fund a joint effort at AI alignment, safety, and certification research. Per their stated objective of having "AI research and development (...) refocused on making today's powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal." Also, loyal? What a strange word to use here.

Re: Pause Giant AI Experiments: An Open Letter

#660

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

"Virtue signalling"? Please. There are a lot of very smart experts on that signatory list who definitely don't need to, or care about, virtue signalling. Fine, ignore Musk's signature, but I don't think luminaries like Stuart Russell, Steve Wozniak, Jaan Tallinn or John Hopfield are doing this for "virtue signalling". You can fairly argue that this will be ineffective, but a lot of experts in this field have real, "h…

"Even Sam Altman"? "Especially Sam Altman", you mean?

While regulations might slightly impact OpenAI's bottom line, they can ultimately prove advantageous for large corporations like them by addressing their primary concern: the threat of competition. By raising barriers to entry, regulatory measures would help solidify OpenAI's market position and maintain its dominance.

Post reply on HN