21–30 of 183 posts
I think AGI is something worth working towards (even though many will make fun of you for even dreaming about it). But I want to know how much you need to sacrifice compared to working a cushy job at some big corp.
to me this is such a waste of resources, trying to build safety for something that doesn't exist and is highly likely to not truly exist for a loooong time.
Prioritizing safety results in a different vantage point on AI/ML/RL. Ensuring safety includes, as a sub-task, really understanding the mathematical foundations of new algorithms and techniques. In some sense, safety research is one way of motivating basic science on AI.
Managed well, a research program on safe AI is a "waste of resources" only in the same way that any basic science is a "waste of resources".
>> "We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that _safety and security concerns will reduce our traditional publishing in the future_, while increasing the importance of sharing safety, policy, and standards research." This seems like the key disclosure statement. I never reconciled how sharing A[G]I te…
To me this doesn't mean "Sharing traditional AI research is dangerous," but rather "Our efforts may be best spent on less popular topics such as safety, policy, and standards research, so you may see us do less traditional AI research." This seems in line with their stated goals around safety.
"To be effective at addressing AGI's impact on society, OpenAI must be on the cutting edge of AI capabilities -- policy and safety advocacy alone would be insufficient."
OpenAI definitely don't plan to do less traditional AI research. What they plan to do is to publish less.
to me this is such a waste of resources, trying to build safety for something that doesn't exist and is highly likely to not truly exist for a loooong time.
Also, safety is an easily addressable issue when the system is truly intelligent. When the systems are dumb and statistical in nature, a lot of work is done on 'safety' as a pseudo-intelligent-control system for an otherwise dumb black box
> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. Wouldn't it be much more likely that a non-value-aligned project comes close first? Wouldn't the Google/Apple/Microsofts of the w…
What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all.
to me this is such a waste of resources, trying to build safety for something that doesn't exist and is highly likely to not truly exist for a loooong time.
> trying to build safety for something that doesn't exist and is highly likely to not truly exist for a loooong time. Prioritizing safety results in a different vantage point on AI/ML/RL. Ensuring safety includes, as a sub-task, really understanding the mathematical foundations of new algorithms and techniques. In some sense, safety research is one way of motivating basic science on AI. Managed well, a research progr…
They didn't even mention several contingencies that, given the rest of the document, should certainly have been addressed: 1) Will they cooperate with aliens who offer humans AGI? 2) If a time traveler hands them AGI invented in the future, will they destroy it? 3) Do they support or oppose human/AGI marriage? How will they respond if one of their employees falls in love with an AGI and they plan to elope? Also, in t…
AI, AGI, and real intelligence all learn from actions and feedback. Looking at simple analogs from animal and human counterparts, setting boundaries and teaching beneficial rules, called morals, works somewhat in non-zero sum environments, but inevitably requires policing when the environment turns competitive. Safety in any case would require Intelligence-proof fencing and a really big stick even the most resource-r…
AGI/Real Intelligence are far different animals than Weak AI and would require far less "safety" and policing. Real Intelligence is a phenomenon that exists on a scale of sorts that many never achieve in its higher forms. It is in lower forms that intelligence lends itself to destructive ends via ignorance.
Attack vectors on a formalized Intelligence/AGI system can be severely restricted using very sensible/affordable approaches. The over complication and pinning of this as a theoretical problem centers on a number of people's desires to profit immensely from FUD.
Overall, AGI exists in a functional form today and has been executed in an online environment. It is secured via physically restricted in-band and out-of-band channels.