381–390 of 414 posts
Earlier quoted context omitted.
You seem to misunderstand what an international ban treaty is. Nuclear proliferation ban included non signatories etc.
Are you talking about the NPT? Can you detail the military strike options it allows if a non-party acquires nuclear weapons?
Earlier quoted context omitted.
> the country that will have the most powerful AI first will be the leader in everything Have you been listening to those podcasts for business leaders that take the idea of AI-powered digital transformation seriously? AI certainly can change things, but I find rhetoric like this to be massively overhyped.
For most of history, Japan has been existed as a small island next to the mighty kingdom of China. Even its name, "Land of the Rising Sun," refers to its relation as a nation east of Chinaa. Then along came the industrial revolution. Soon Japan was out conquering China, despite China having a much larger army. AI will be much faster, and much, much more powerful than the Industrial Revolution. If Switzerland is the f…
I'm an AI skeptic. Most warnings feel more like science fantasy, specifically of the retro-futurism genre.
In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…
EY’s arguments don’t seem to take physics into account. That’s why they are not taken very seriously. To many it sounds like: volcanos are dangerous. What if there were a billion volcanos? We would all die. Everything individual sentence is true. Volcanos are dangerous. If there were a billion of them the world might be uninhabitable. The thing preventing a billion volcanos is like… thermodynamics
The claim that computers just can’t do X because “physics” looks weaker every day. Intelligence isn’t magic, hardware today seems more than capable. I think Carmack is probably right, it’s not a hardware constraint at this point - it’s a software intuitive leap.
Earlier quoted context omitted.
Agreed point 2 seems to be the crux. An important concept behind this is Omohundro's Basic Drives. Any maximizing agent with a goal will try to acquire more resources, resist being shut off, resist having its goal changed, create copies of itself, and improve its algorithm. If it is possible to maximize its goal in a way that will not guarantee humanity's flourishing, then we all die, guaranteed. If you want somethin…
Yes point 2 is the weak one. Aren't humans/biological life a counterexample? Simple bacteria are clearly maximizing agents, and cyanobacteria did infact almost destroy all life on earth by filling the entire planet with toxic oxygen. We've known how to improve ourselves via selective breeding yet vanishingly few humans are proponents. We and other intelligent animals have a wide variety of goals, and even share food…
Intellectual capability and alignment are orthogonal - the former doesn’t get you the latter for free.
Most people new to this issue don’t intuitively grasp this at first.
Earlier quoted context omitted.
Are you talking about the NPT? Can you detail the military strike options it allows if a non-party acquires nuclear weapons?
Looking Iran,Iraq,any country that has a bioweapons program etc. It's pretty trivial to understand what Eliezer meant if you truly want to.
Earlier quoted context omitted.
Yes point 2 is the weak one. Aren't humans/biological life a counterexample? Simple bacteria are clearly maximizing agents, and cyanobacteria did infact almost destroy all life on earth by filling the entire planet with toxic oxygen. We've known how to improve ourselves via selective breeding yet vanishingly few humans are proponents. We and other intelligent animals have a wide variety of goals, and even share food…
Natural selection and biology have different constraints that limit things (and even then often selection can cause extinction) Intellectual capability and alignment are orthogonal - the former doesn’t get you the latter for free. Most people new to this issue don’t intuitively grasp this at first.
Remember that ML models are also subject to selection. If they're not useful/beneficial to us, we stop running them.
I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…
[flagged]
Sure, point me to the research papers and discourse on the solution you propose.
I feel like if I actually believed that the death of all biological life was imminent and that the only way to forestall it was to bomb data centers, I'd spend a lot more time planning how to bomb data centers and a lot less time curating Shoggoth memes on my web forum. On the other hand, if I were more concerned with self-aggrandizement, writing doomsaying thinkpieces for Time is exactly what I'd spend my time on.
If you believed that the death of all biological life was imminent and that the only way to forestall it was to bomb data centers, and you knew that there were far more data centers than you personally could bomb, what would you do?