771–780 of 873 posts
Earlier quoted context omitted.
We have a decent grasp of the odds of an asteroid extinction event in a way that we don't for AI.
And that makes taking that extinction risk everyday better?
We can work on it in the long term and things like developing safe AI may have more impact on mitigating asteroid risk than working on scaling up existing nuclear, rocket, and observation tech to tackle it.
Earlier quoted context omitted.
No one's calculator needs to. The point was to reply to the notion that there is any possible way the "war on general-purpose computation" actually has any shot of actually stopping general-purpose computation to make a point about how hard limiting computation is in general. > It will only happen in a datacenter with an ocean of H100 GPUs. This computing power does not materialize out of thin air. It can be monitore…
There are physical limits to what can make an effective and efficient computational substrate. There are physical limits to how fast/cheap these GPUs can be made. It is highly doubtful that some rogue nation is going to invent some entirely unknown but equally effective computational method or substrate. Controlling the source of the known substrate is possible and effective. Most nations aren't in a position to just…
As it is, we keep seeing researchers with relatively modest funding steadily driving down the amount of compute required for equivalent quality models month by month. Couple that with steady improvements in realigning models for peanuts reducing the need to even start from scratch.
There's enough room for novel reductions in compute to keep the process of cost reducing training going for many years.
As it is, I can now personally afford to buy hardware sufficient to train a GPT3 level model from scratch. I'm well off, but I'm not that well off. There are plenty of people just on HN with magnitudes more personal wealth, and access to far more corporate wealth.
Even a developing country can afford enough resources to train something vastly larger already today.
When your premise requires fictional international monitoring agencies and fictional agreements that there's no reason to think would get off the ground in anything less than multiple years of negotiations to even create a regime that could try to limit access to compute, the notion that you would manage to get in place such a regime before various parties will have stockpiled vast quantities in preparation is wildly unrealistic.
Heck, if I see people start planning something like that, I'll personally stockpile. It'll be a good investment.
If anything is gaslighting, it's pushing the idea it's possible to stop this.
Earlier quoted context omitted.
For frontier AI models you don't need a few GPUs or a small cluster. You need thousands of them, you need them all connected with high bandwidth and you need a lot of power. It's not something you can do in your garage or undetected. There is some risk that over time people will be able to do more with less, and eventually you can train an AGI with a few dozen A100s. If that happens I agree there's nothing you can do…
What is the difference between a weather model, a nuclear bomb simulation, and training a LLM? At the end of the day all these are just calculations, of which you need thousands of processors to do with high accuracy. Pretty much what you're suggesting is a global police force to ensure we're performing the 'correct calculations'. Having to form an authoritarian world computer police isn't much of a solution to the p…
It would be better if we didn't need to do this, but I don't see a less intrusive way.
Of course, if you don't think there is a real existential threat then it's pointless and bad to do this, it all depends on that premise.
Earlier quoted context omitted.
I lost total interest in every “Stop” argument when Elon Musk started a company to develop AGI during the pause that he signed on to and promoted. Yann LeCunn said that this was just an attempt to seize market share among the big companies and he couldn’t have been more right.
That is transparently what this letter is for most of its signatories. They lost the race (in one way or another) and they want time to catch back up. This is as true for the academics who made big contributions in the past but missed out on the latest stuff as it is for Elon Musk.
Earlier quoted context omitted.
That is transparently what this letter is for most of its signatories. They lost the race (in one way or another) and they want time to catch back up. This is as true for the academics who made big contributions in the past but missed out on the latest stuff as it is for Elon Musk.
The joke will be on them when our open-source powered mechwarriors made of bulldozer parts come for them.
Earlier quoted context omitted.
I disagree that it is “human-like” in any meaningful sense. It’s completely different. We’ve been trying to build AI for a long time yes, but we only just figured out how to build AI that actually works.
> It’s completely different. In what way? The implementation is completely different, if that is what you mean, but the way humans interpret AI it is the same as far as I can tell. Hell, every concern that has ever been raised about AI is already a human-to-human issue, only imagining that AI will take the place of one of the humans in the conflict/problem. > but we only just figured out how to build AI that actually…
I wouldn't really say an AI beat a human chess player in 1967--I'd say a computer beat a human chess player. In the same way that computers have for a long time been able to beat humans at finding the square roots of large numbers. Is that "intelligence"?
I grant you though that a lot of this comes down to semantics.
Earlier quoted context omitted.
> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…
Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.
"${my favorite work of fiction} mentioned/alluded to it too!" does not make that work of fiction equivalent to a serious take on the topic in real-world context.
Earlier quoted context omitted.
> Arguably, it’s even less of a real world risk because it’s a common sci-fi trope. It would be nice if the world worked that way. Then we could diminish any risk just by writing a lot of scifi about it :)
That's exactly how it works. Invest thought into it before it happens, then when it becomes an issue, there's already awareness and possibilities have been mapped out.
Earlier quoted context omitted.
Why does extinction matter? If we lose civilization now, we'll be stuck at pre-industrial levels possibly for geological timescales, as we've mined and burned all the high-density energy sources you can get at without already being a high-tech industrial society.
I don't think that would necessarily limit us. Windmills and water mills can be made with wood, stone, and the small quantities of iron that pre-industrial societies could produce.
Earlier quoted context omitted.
Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it poses a greater and more immediate existential risk to humanity than AI ever could; it is largely mitigated (though still a bigger X-Risk than AI) because no one with the capacity to implement it in any substantive way takes him seriously at the moment. But the people selling AI X-Risk that anyone with any poli…
> Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it Could you provide a link to where he said this?
https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...
The moratorium on new large training runs needs to be indefinite and worldwide. There can be no exceptions, including for governments or militaries. If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S. and in China and on Earth.
.
.
.
Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make immediate multinational agreements to prevent the prohibited activities from moving elsewhere. Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.