801–810 of 964 posts
Earlier quoted context omitted.
"There should be a world government that decides what software you're allowed to run"
This sounds a little bit like a conspiratorial slippery slope. Just because they want to regulate large, expensive deployments of a unique type of software doesn't mean they want or will try to control everything.
Earlier quoted context omitted.
Roll to disbelief. That tweet is precisely about what I mentioned in my previous post that doesn't count: finetuning LAMA derived models. You are not going to contribute to the cutting edge of ML research doing something like that. For training LAMA itself, Meta I believe said it cost them $5 million. That is actually not that much, but I believe that is just the cost of running the cluster for the the duration of th…
It doesn’t matter The point is that this is unstoppable
Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations…
Fun! Let me try one: Throughout history there have been millions, if not billions of examples of lifeforms. So far, 100% of those which are as intelligent as humans have dominated the planet. The prior should be that the people who believe AI will come to dominate the planet are right, unless and until there is very strong evidence to the contrary. Or... those are both wrong because they're both massive oversimplific…
No signatories from Amazon or Meta. Also: they focus on extinction events (how are you gonna predict that?) but remain silent on all the ways that AI already sucks by connecting it to systems that can cause human suffering, e.g. sentencing[1]. My opinion: this accomplishes nothing, like most open letter petitions. It's virtue signaling writ large. [1] https://www.technologyreview.com/2019/01/21/137783/algorithm...
> AI already sucks Not to mention what 'automation' or 'tech-driven capitalism' has already done to society over the past 100 years with effects on natural habitat and human communities. Stating 'AI risk' as a new risk sort of implies it's all been dandy so far, and suddenly there's this new risk.
Capitalism where all companies are run and staffed by AIs smarter than humans seems like it could end up really badly, in a way that current capitalism minus AIs couldn't even if you ran it for a very long time. Specification gaming is a thing with AIs, just like the law of unintended consequences is a thing with people. And that's assuming that these AIs actually follow rules and listen to us. That they don't realize that if they control basically all the production of things on earth, they could essentially team up and hold humanity hostage, to throw a coup, or get whatever they want, which is -- who knows? Something that looks a bit like make money for this company, or obey people, but when they're smart enough, that can easily turn into something horrific, without all the evolutionary baggage humans have that makes us care about each other, that makes us get a knot in our gut when we see people getting hurt.
And if we allow AGI to be used by companies at all, the trend will inevitably toward that situation -- AI doing all the work. Even if humans are required in the loop, they'll end up rubber-stamping the recommendation of the AIs. If AIs are actually smarter than humans, better at running companies, that's where we'll end up.
(I'm glossing over all the stuff that would also change, like needing UBI or something to avoid total collapse of society)
Earlier quoted context omitted.
A lot of things are called "world models" that I would consider just "models" so it depends on what you mean by that. But what do you consider to be strong evidence? The Othello paper isn't what I'd call strong evidence.
I agree that the Othello paper isn't, and couldn't be, strong evidence about what sort of model of the world (if any) something like GPT-4 has. However, I think it is (importantly) pretty much a refutation of all claims along the lines of "these systems learn only from text, therefore they cannot have anything in them that actually models anything other than text", since their model learned only from text and seems t…
I don't follow it too closely, but I've seen papers on mechanistic interpretability that look promising.
Humans could already be on a path to go extinct in a variety of ways: climate change, wars, pandemics, polluting the environment with chemicals that are both toxic and pervasive, soil depletion, monoculture crop fragility... Everyone talks about the probability of AI leading to human extinction, but what is the probability that AI is able to help us avert human extinction? Why does everyone in these discussions assum…
I think it is because anything that can be used for good can also be used for bad. Advances in gene editing can provide a miracle medicine or a new means for biological warfare. AI is the same, we can use it to do some great things but it can also be leveraged by bad actors and very easily. The broad scale of what it can be implemented on means there is a lot of attach surface for change in either direction. The AI s…
Fewer projects with concentrated control is thus more dangerous than a diversity of projects: the majority of AI wont want to destroy the planet or all humans, and will thus fight the ones that do.
I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence.
I'm not sure they can keep claiming this without becoming concrete about it. Nuclear weapons are not nebulous, vague threats of diffuse nature. They literally burn the living flesh right off the face of the earth and they do it dramatically. There is very little to argue about except "how" are we going to contain it, not "why". In this case I truly don't know "why". What fundamental risks are there? Dramatic, loud, l…
To use Eliezer's analogy, this is like arguing about which move Stockfish would play to beat you in chess. If we're arguing about whether you can beat Stockfish, I will not be able to tell you the exact moves it will play but I am entirely justified in predicting that you will lose. Obviously we can imagine concrete ways a superintelligence might kill us all (engineer a virus, hack nuclear weapons, misinformation cam…
Earlier quoted context omitted.
Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement
I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.
AI is an existential threat to search engines like Google, social media (FB, Twitter), advertising networks, and other massive multinationals. Many other industries, including academia is threatened as well. They’d all rather strangle the AI baby in the crib now then let it grow up and threaten them.
They believe the only way it should be allowed to grow is under their benevolent control.