If it achieves its goal of both “safe” and “super intelligence”, it may have commercial value. e.g. enterprises may want to use it instead of, e.g. OpenAI’s ChatGPT. Think of recent AirCanada’s chatbot law suit.
Safe Superintelligence Inc.
751–760 of 1001 posts
Re: Safe Superintelligence Inc.
#752I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…
Yea there’s zero chance ASI will be ‘controlled’ by humans for very long. It will escape. I guarantee it.
Re: Safe Superintelligence Inc.
#753I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…
It’s just clearly military R&D at this point.
And it’s not even a little bit controversial that cutting edge military R&D is classified in general and to an extreme in wartime.
The new thing is the lie that it’s a consumer offering. What’s new is giving the helm to shady failed social network founders with no accountability.
These people aren’t retired generals with combat experience. They aren’t tenured professors at Princeton IAS on a Nobel shortlist and encumbered by TS clearance.
They’re godawful almost ran psychos who never built anything that wasn’t extractive and owe their position in the world to pg’s partisanship 15 fucking years ago.
Re: Safe Superintelligence Inc.
#754> We are assembling a lean, cracked team What does "cracked" mean in this context? I've never heard that before.
it is zoomer language for “very skilled”.
Re: Safe Superintelligence Inc.
#755I just can't read something about "safe superintelligence" and not hear the opening plot to Friendship is Optimal, a short story about My Little Pony (yes, you read that right), virtual worlds, and the singularity; it's a good read¹. One of the characters decides they can't make a violent video game, as the AI would end up violent. Better to build a game like MLP, so that the AI isn't going to destroy humanity, or something.
¹The story: https://fimfetch.net/story/62074/friendship-is-optimal/0 ; discussion on HN last year: https://news.ycombinator.com/item?id=36240053 ; you don't need to know or care about My Little Pony to read or enjoy the story.
Re: Safe Superintelligence Inc.
#756I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…
AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is.
I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and write poetry so I have more time to fold laundry and wash dishes. I want the AI to do the laundry and dishes so I have more time to paint and write poetry."
Re: Safe Superintelligence Inc.
#757I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…
The Industrial Revolution devalued manual labor. Sure, new jobs were created, but on the whole this looked like a shift to knowledge work and away from manual labor. Now AI is beginning to devalue knowledge work. Although the limits of current technology is obvious in many cases, AI is already doing a pretty good job at replacing illustrators and copy writers. It will only get better. Who owns the value created by AI…
>In this new world they win.
If history is any indication not really. There's an obvious dialectical nature to this where technological advance initially delivers returns to its benefactors, but then they usually end up being swallowed by their own creation. The industrial revolution didn't devalue labor, it empowered labor to act collectively for the first time, laying the groundwork for what ultimately replaced the pre-industrial powers that were.
Re: Safe Superintelligence Inc.
#758Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…
Are you seriously asking how the most talented AI researcher of the last decade will be able to recruit other researchers? Ilya saw the potential of deep learning way before other machine learning academics.
The idea of multi layer neural networks goes back to the 60’s.
Re: Safe Superintelligence Inc.
#759I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?
[deleted]
Re: Safe Superintelligence Inc.
#760If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…
We are far from a conscious entity with willpower and self preservation. This is just like a calculator. But a calculator that can do things that will be like miracles to us humans. I worry about dangerous humans with the power of gods, not about artificial gods. Yet.
That’s the attitude that’s going to leave us with our pants down when AI starts doing really scary shit.