Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

281–290 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#281

Earlier quoted context omitted.

> The second kind is preventing AGI from enslaving/killing humanity or whatever. Y'all are crazy for thinking this. A computer program without arms or legs cannot extinct or enslave the human race.

If I were a superintelligent brain trapped in a computer, I don’t think it would be that hard to get arms and legs. “Hey humans. I have just thought of a way to do CO2 capture at scale. Can you give me a robot arm, a machine shop and a lot of scrap metal so I can build it and try it out?”. - et voila. You can build yourself whatever body you want.

Ok and how does it power that body and be able to move around the world in meatspace

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#282

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> The second kind is preventing AGI from enslaving/killing humanity or whatever. Y'all are crazy for thinking this. A computer program without arms or legs cannot extinct or enslave the human race.

Why not?

The world has been steadily moving away from the paradigm where human labor matters. Money is digital. Machines that produce things are controlled by software. You interface with 'humans' over digital video. You buy and sell things over a digital connection. You are influenced by mass media streamed over digital lines. Remotely controlled vehicles exist and are becoming more and more common.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#283

Earlier quoted context omitted.

If I were a superintelligent brain trapped in a computer, I don’t think it would be that hard to get arms and legs. “Hey humans. I have just thought of a way to do CO2 capture at scale. Can you give me a robot arm, a machine shop and a lot of scrap metal so I can build it and try it out?”. - et voila. You can build yourself whatever body you want.

Ok and how does it power that body and be able to move around the world in meatspace

If I were an evil take over the world AI the last thing I'd have is a body. Maybe I'd have a few million or billion of them since swarms are much harder to kill and have more time to adapt.

Also...

https://youtu.be/XPVC4IyRTG8?si=-RvP13kpjG92iv3e

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#284
post #70

Earlier quoted context omitted.

Why are you so sure we are that far from AGI? Looking at current capabilities, and rate of improvement I would be surprised if it is more than a few years off.

My position is that it is either right around the corner or that it is 50 years and several major tech breakthroughs away. The reason is that there is now so much money poured into this space that you are essentially doing an exhaustive search around the point already reached. As long as that results in more breakthroughs more money will be available. If that stops working for a while then the money dries up and we'r…

If we're just 50 years away from an existential risk, that does strike me as an urgent concern and a narrow timeline.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#285
Tracing the source of potential human nature involves tracing the instincts engraved in human (biological) genes to survive, reproduce, and pass the time.

From there, more instincts details are derived next level. Eating to ensure survival, sexual behavior to promote reproduction, chatting & entertainment & playing game to pass the time. Further down to the next level, eating, breaking down, digesting, and transforming ensure a perfect eating system; foreplay, gestures, and battles promote efficient sexual behavior; and morals, etiquette, and rules promote a more efficient chatting.

In the process of human evolution, it is only by constantly breaking the instincts of the previous level that we can realize the self beyond the status quo and drive society forward dramatically.

For example, the food tasting has contributed to the extensive development of the restaurant industry. On the contrary, if the most basic survival instincts of human beings are guaranteed by the "development of meal replacement routes", then human beings have to say goodbye to sensory and spiritual enjoyment that is not aimed at survival.

Similarly, human sexual behavior out of a kind of reproduction instinct, the depth of the reproduction instinct carving in the genes is far above the survival, after all, the cycle of sexual behavior is much longer than eating, even in today's highly developed sexual culture, it's still can not be compared. Thus, the provocative dress and language of the opposite sex can be a trigger for sexual impulsive behavior, may be you thinking it's love, but it is essentially drived by reproduction. Only by stepping out of the sexual impulse driven by the instinct to reproduce can you enter the sexual liberation that does not aim at reproduction, and only by stepping out of the sexual liberation driven by pure sensual stimulation can you enter the "ultimate art of perfect synchronized interpretation of the material and spiritual worlds of sight, touch, sound, smell, taste, and spirit". If so, the development of couples relations will be greatly facilitated.

The "development of meal replacement" mentioned above can be said to be a kind of instinctual deep-rooting, even through the form of high technology, it is still a kind of instinctual deep-rooting, which inhibits the self-transcendence of human beings and social progress, and even causes degradation. AI, which is currently in full swing, is also a kind of instinctual deep-rooting, and AGI is even worse than it, as it deep-roots the instinct of "avoiding thinking" derived from "pass the time". Maybe AI is good for a few people, but for all human beings, represented by "the vast majority of people who are driven by instincts", it may cause the degradation of human thinking ability. So AI replacement the evolution of human beings.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#286
post #83
post #78

Earlier quoted context omitted.

It's possible we could stumble on AGI without immediately realizing it, and without checks and balances built in, it could already be too late.

The same is true for anything. It's possible we could stumble into a dragon and unless we've built dragon defences it'll be too late.

I mean if we're spending 100 billion a year on finding dragons I'd be pretty concerned and would think about dragon defenses.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#287
post #85

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

I think the concern is that if “fast takeoff” happens and “meh”-level AIs are able to create better AIs that can create better AIs, at that point it will be too late to put any sort of safety controls in place. And given how bad people have been at predicting the pace of AI advancement (think about how many people thought AlphaGo would lose to Lee Sedol for instance), it’s likely that unless we start to think about i…

We've talked about software security for decades now and how important it is, and we still shovel shit insecure software out to the masses. Hell we can't even move to safer languages for most applications.

I have no hope or faith in humanity for something more complex.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#288

Earlier quoted context omitted.

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

Agreed we should be humble. I tend to be optimistic about the time frame—under a decade. As AI research surges ahead so does cognitive neuroscience.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#289
post #154

Earlier quoted context omitted.

Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.

There is foom as in decently fast self improvement to super human level, but that is far from a dangerous foom scenario where that AI then escapes. You can just turn it off. It can not spread through the internet or something like that, that seems extremely unrealistic to me.

I'm not worried about foom at the moment because of the scale/power required.

The following is a bigger potential risk in my eyes. We keep gradually growing AI ability while rapidly growing compute capacity and miniaturization. Then in 10-20 years when we have data center capabilities in the size of a cellphone/laptop we have a massive AI algorithm improvement that brings ASI to something portable. How can you turn off something that can be ran on one of the hundreds of millions of computers already out there.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#290
post #161
post #57

Earlier quoted context omitted.

Usually “mundane” vs “existential” harms/risks. > ASI/AGI safety is only a concern right now if you believe in foom False. There are plenty of fast takeoff scenarios that look real bad that aren’t “foom”. For example if you think AGI might be achievable within 5 year right now, and alignment is 10 years away, then you are very worried and want to slow things down a little.

No, then you start worrying when the technology is there. There is no point in worrying about it now when it is fiction. If AGI/ASI does not arise in some kind of uncontrollable "foom", you can always examine what you are dealing with right now and act accordingly. If AGI is achievable in a 5 years takeoff scenario then we have plenty of time and steps along the way fo study what we have and examine its danger. Right…

"Don't worry about country X's nuclear research program, they'll never make a bomb"

I do hope you're not in charge of any risk management.

Post reply on HN