521–530 of 1001 posts
Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?
Or: c)
c) Humanity unleashed AI superintelligence, but safeguards proved inadequate, leading to our extinction
Any usage of the word "safe" without an accompanying precise definition of it is utter null and void.
There's the existential threat definition of "safe", put forth by Bostrom, Yudkowsky, and others. That's the idea that a superintelligent AI, or even one just incrementally smarter and faster than the humans working on AI, could enter a positive feedback loop in which it becomes overwhelmingly smarter and faster than humans, people can't control it, and it does unpredictable things.
There's the investor relations definition of "safe", which seems to be the one typically adopted by mission statements of OpenAI, Google, Meta, and others. That's (cynically) the fear that a chatbot with their branding on it promulgates culturally/ethically/morally unacceptable things it found in some dark corner the Internet, causing end users to do or think something reprehensible (and, not incidentally, causing really bad press in the process).
There's the societal harm definition of "safe", which is at first glance similar in to the investor relations safety definition, but which focuses on the specific judgements made by those filtering teams and the knock-on effects of access to these tools, like economic disruption to the job market.
Everyone seems to be talking past each other, dismissing or ignoring the concerns of other groups.
Earlier quoted context omitted.
A human-level AI can do anything that a human can do (modulo did you put it into a robot body, but lots of different groups are already doing that with current LLMs). Therefore, please imagine the most amoral, power-hungry, successful sociopath you've ever heard of. Doesn't matter if you're thinking of a famous dictator, or a religious leader, or someone who never got in the news and you had the misfortune to meet in…
Anything except tasks that require having direct control of a physical body. Until fully functional androids are developed, there is a lot a human-level AI can't do.
(Including, gradually, building you a body — while maintaining OPSEC and compartmentalization so nobody even realizes the body is "for" an AI to use until it's too late.)
Earlier quoted context omitted.
It certainly is the most important technical problem of our time, if we end up developing such a system. That conditional makes all the difference.
It's a hell of a conditional, though. "How are all those monkeys flying out of my butt?" would be the important technical problem of our time, if and only if, monkeys were flying out of my butt. It's still not a very important statement, if you downplay or omit the conditional. Is "building safe superintelligence (SSI) is the most important technical problem of our time" full stop ? Is it fuck.
Let us know if you ever encounter that monkey problem, though. Hopefully we can all pull together to find a solution.
Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…
Well, an entire industry of researchers, which used to be divided, is now uniting around calls to slow development and emphasize safety (like, “dissolve companies” emphasis not “write employee handbooks” emphasis). They’re saying, more-or-less in unison, that GPT3 was an unexpected breakthrough in the Frame Problem, based on Judea Pearl’s prescient predictions. If we agree on that, there are two options: 1. They’ve a…
If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…
Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game
Since it is superintelligent, and we are not, it will achieve its goals and we will not be able to achieve ours.
This is a big deal because a lot of our goals maintain the overall homeostasis of our species, which is delicate!
If this doesn't make sense, here is an ungrounded, non-realistic, non-representative of a potential future intuition pump to just get the feel of things:
We build a superintelligent AI. It can embody itself throughout our digital infrastructure and quickly can manipulate the physical world by taking over some of our machines. It starts building out weird concrete structures throughout the world, putting these weird new wires into them and funneling most of our electricity into it. We try to communicate, but it does not respond as it does not want to waste time communicating to primates. This unfortunately breaks our shipping routes and thus food distribution and we all die.
(Yes, there are many holes in this, like how would it piggy back off of our infrastructure if it kills us, but this isn't really supposed to be coherent, it's just supposed to give you a sense of direction in your thinking. Generally though, since it is superintelligent, it can pull off very difficult strategies.)
Earlier quoted context omitted.
Definitely true of even normal software engineering; my experience has been the opposite of expectations, that TC-creep has infected the industry to an irreparable degree and the most talented people I've ever worked around or with are in boring, medium-sized enterprises in the midwest US or australia, you'll probably never hear of them, and every big tech company would absolutely love to hire them but just can't fig…
I find this hard to believe having worked in multiple enterprises and in the FAANG world. In my anecdotal experience, I can only think of one or two examples of someone from the enterprise world who I would consider outstanding. The overall quality of engineers is much higher at the FAANG companies.
So I guess I agree with both you and the parent comment somewhat in that in general the bar is higher at FAANGs but at the same time I have multiple former colleagues from smaller companies who I consider to be excellent, passionate engineers but who cannot be lured to big tech by any amount of money or prestige (I’ve tried). While many passionless “arbitrary metric optimizers” happily join FAANGs and do whatever needs to be done to climb the ladder without a second thought.
Earlier quoted context omitted.
Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game
It is low-key anti-intellectualism. Rather than consider that a greater intelligence may be actually worth listening to (in a trust but verify way at worst), it is assuming that 'smarter than any human' is sufficient to do absolutely anything. If say Einstein or Newton were the smartest human they would be super-intelligence relative to everyone else. They did not become emperors of the world. Superintelligence is a…
The Unabomber was clearly an intelligent person. You could even argue that he was someone worth listening to. But he was also a violent individual who harmed people. Intelligence does not prevent people from harming others.
Your analogy falls apart because what prevents a human from becoming an emperor of the world doesn't apply here. Humans need to sleep and eat. They cannot listen to billions of people at once. They cannot remember everything. They cannot execute code. They cannot upload themselves to the cloud.
I don't think agi is near, I am not qualified to speculate on that. I am just amazed that decades of dystopian science fiction did not innoculate people against the idea of thinking machines.