What exactly is "safe" in this context, can someone give me an eli5? If it's "taking over the world" safe, does it not mean that this is a part of AGI?
It's "not killing every human" safe.
Safe Superintelligence Inc.
421–430 of 1001 posts
Re: Safe Superintelligence Inc.
#422Any usage of the word "safe" without an accompanying precise definition of it is utter null and void.
“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” https://www.safe.ai/work/statement-on-ai-risk , signed by Ilya Sutskever among others.
I clicked, hoping that "human extinction" was just the worst thing they were against. But that's the only thing. That leaves open a whole lot of bad stuff that they're OK with AI doing (as long as it doesn't kill literally everyone).
Re: Safe Superintelligence Inc.
#423Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…
Is safe AI really such a genie out of the bottle problem? From a non expert point of view a lot of hype just seems to be people/groups trying to stake their claim on what will likely be a very large market.
Re: Safe Superintelligence Inc.
#424Cant wait for the SS vs OpenAI peace wars
Just a joke , congrats to Ilya
Re: Safe Superintelligence Inc.
#425What does "safe" mean?
1. Will not produce chat results which are politically incorrect and result in publicity about "toxic" comments?
2. Will not return false factual information which is dangerously wrong, such as that bad recipe on YC yesterday likely to incubate botulism toxin?
3. Will not make decisions which harm individuals but benefit the company running the system?
4. Will not try to take over from humans?
Most of the political attempts focus on type 1. Errors of type 2 are a serious problem. Type 3 errors are considered a feature by some, and are ignored by political regulators. We're not close to type 4 yet.
Re: Safe Superintelligence Inc.
#426Earlier quoted context omitted.
I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which,…
i am very skeptical of narratives saying that young boys or men are more misogynistic than in the past. we have a cognitive bias towards thinking the past is better than it was, but specifically on gender issues i just do not buy a regression
I mean, I don't know if it's better or worse than it was. I do know that it's bad, thanks to tons of studies on the subject covering a wide range of little kids who watch shitheads like Andrew Tate, Fresh & Fit, etc. Most grow out of it, but speaking as someone who did, I would be a much better and happier person today if I was never exposed to that garbage in the first place, and it's resulted in stunted social skills I am still unwinding from in my thirties.
This shit isn't funny, it's mental poison and massive social media networks make BANK shoving it front of young men who don't understand how bad it is until it's WAY too late. I know we can't eliminate every kind of shithead from society, that's simply not possible. But I would happily settle for a strong second-place achievement if we could not have companies making massive profits off of destroying people's minds.
Re: Safe Superintelligence Inc.
#427Earlier quoted context omitted.
If you're working on superintelligence I don't think you'd be worried about not getting credit due to a lack of publications, of all things. If it works, it's the sort of thing that gets you in the history books.
Not sure about that. It might get Ilya in the history books, and maybe some of the other high profile people he recruits early on, but a junior researcher/developer who makes a high impact contribution could easily get overlooked. Whereas if that person can have their name as lead author on a published paper, it makes it much easier to measure individual contributions.
There is a human cognitive limit to the detail in which we can analyze and understand history.
This limit, just like our population count, will not outlast the singularity. I did the math a while back, and at the limit of available energy, the universe has comfortable room for something like 10^42 humans. Every single one of those humans will owe their existence to our civilization in general and the Superintelligence team in specific. There'll be enough fame to go around.
Re: Safe Superintelligence Inc.
#428Given that GenAI is a statistical approach from which intelligence does not emerge as ample experience proves, does this new company plan to take a more human approach to simulating intelligence instead?
Lossy compression of all world information results in super intelligence....
Thats the whole eureka thing to understand... To compress well, you need to understand. To predict the next word, you need to undestand the world.
Ilya explains it here: https://youtu.be/GI4Tpi48DlA?t=1053
Re: Safe Superintelligence Inc.
#429Earlier quoted context omitted.
It'd be naive if it wasn't literally a standard point that is addressed and acknowledged as being a major part of the problem. There's a reason OpenAI's charter had this clause: “We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop c…
How does that address the issue? I would have expected them to do that anyhow. Thats what a lot of businesses do: let another company take the hit developing the market, R and D, and supply chain, then come in with industry standardization and cooperative agreements only after the money was proven to be good in this space. See electric cars. Also they could drop that at any time. Remember when openAI stood for openso…
Really, you think Ford is dropping their electric car manufacturing in order to assist Tesla in building more gigafactories?
> Remember when openAI stood for opensource?
I surely don't, but maybe I missed it, can you show me?
https://web.archive.org/web/20151211215507/https://openai.co...
https://web.archive.org/web/20151213200759/https://openai.co...
Neither mention anything about open-source, although a later update mentions publishing work (“whether as papers, blog posts, or code”), which isn't exactly a ringing endorsement of “everything will be open-source” as a fundamental principle of the organization.
Re: Safe Superintelligence Inc.
#430If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…
If superintelligence can be achieved, I'm pessimistic that a team committed to doing it safely can get there faster than other teams without the safety. They may be wearing leg shackles in a foot race with the biggest corporations, governments and everyone else. For the sufficiently power hungry, safety is not a moat.