Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

191–200 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#191

Post by 21 »

lemming wrote: post #186
nopinsight wrote: post #91

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

To quote Bret Victor (from memory): worrying about general AI when climate change is happening is like standing on the train tracks in the station when the train is rushing in, worrying about being hit by lightning. Work is being done on climate change, but at the current rate, we're way off preventing catastrophic effects. I'd say that our current level of effective effort basically amounts to ignoring the problem.

Climate change is not adversarial, the weather will not try to disrupt and avoid you fixing it.

Fighting a passive adversary is a much easier thing than fighting an active one (especially one smarter than you).

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#192

Post by wodenokoto »

daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

In that scenario, sinking a Chinese ship in Chinese waters is still very problematic.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#193

Post by unityByFreedom »

pure21r wrote: post #178
unityByFreedom wrote: post #146

Earlier quoted context omitted.

> We also don't know how long it will take to solve Provably Safe AI problem. Wouldn't it be wiser to start preparing now? Sure, go for it. I think this has been in the news a lot recently because Musk told government leaders to expect AGI as early as 2030-2040. It's a bit of a ridiculous claim. One of the biggest AI competitions, ImageNet, expects to run for the next 12 years, and that's just for making sense of ima…

So what would be your equivalent position in the 1940s about the nuclear solution? "Don't worry about it, humanity has always prevented slow moving disasters."? Because the question is not about whether one should develop AI or not, but if we should be worried and work actively to minimize risk, as has been done on the nuclear case... So suppose you simulate 100 Earths from today t=0? What are the best strategies to…

Edit: You don't need to paste the same comment all over this thread. Spamming doesn't lead to interesting conversation.

> So what would be your equivalent position in the 1940s about the nuclear solution?

I've seen people throwing that example in this thread and I don't see why.

Multiple countries invested heavily in building the bomb. It was not a question if building a bomb was possible, but rather how soon, and by whom.

> In other words, what are the risks and benefits of adopting a "worried" versus a "don't worry about it" position?

Some people do worry about it, but everyone need not. I don't.

> If there's a possible huge meteor (say 30% chance) of hitting the Earth in about 50 years, would you be "worried"?

Good example. No I wouldn't because I know we have systems monitoring for such an event. If something is coming, we will know about it. I'm not sure how far in advance we would know, but I expect as it got nearer more and more people would do their best to come up with a good solution.

We're no nearer to AGI today than we were when neural networks were developed 30 years ago. Science fiction would have you believe it, but we don't look at SciFi movies of the 50s as particularly prescient, and we shouldn't do so for today's films either.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#194

Post by throwaway47861 »

tanilama wrote: post #85
nopinsight wrote: post #78

Top people including DeepMind's CEO Demis Hassabis and Prof Stuart Russell, a AAAI and AAAS fellow who is a co-author of the AI textbook used at most of top universities, agree that AGI is definitely possible and going to happen. [1] Hassabis also stated in another session that there are probably at least half a dozen mountains to climb before reaching AGI and he would be surprised if it takes more than 20. He also s…

> Top people including DeepMind CEO Demis Hassabis and Prof Stuart Russell, a AAAI and AAAS fellow who is a co-author of the AI textbook most used at top universities, agree that AGI is definitely possible and going to happen. Even though they are high-profile people, in DL, since people still don't know a lot about it, their confidence means nothing. When I was in college, my professors/textbook alike, claimed that…

>Guess what, no one, not even the most optimistic person, would predict it will beat the best human player in 10 years.

This is what I call living in a bubble, dear sir/madam.

I grew in a small town in Eastern Europe (~100k population) and me plus the local 50 or so programmers (using Apple II and 8086-based computers), in 1996, while we were in our teenage years, were painfully aware that games like chess and Go are brute-force-able with some elements of recognizing and memorizing viable strategies (and throwing out the unviable ones early on). And that was what, 15-19 year old late teenagers, and we knew it. So be sure that many people knew it -- they just never chose that area of expertise specifically.

This also says a lot about the quality of the average professors and textbooks, but let's not go off-topic, plus it's a huge discussion area.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#195

Post by pdimitar »

unityByFreedom wrote: post #168
nopinsight wrote: post #161

Earlier quoted context omitted.

For subgoals, there have been a lot of research results on that, mostly in the symbolic AI tradition. Even if an AI does not determine its own ultimate goal(s), it could still cause us a lot harm by creating subgoals that do not align with human values, which no one can articulate in full yet. Overall, the controversy is worth it because the force for progress is strong enough that another AI winter is highly unlikel…

> the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Really disagree here. All it takes is a few well positioned people, like those at Nnaisense, to create companies promising AGI and sucking up investment dollars. We all know greed is rampant in finance, and AI tech is a big word in startups/innovation these days that can land big VC dol…

I see a small controversy in your statements, please help me understand:

> Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives.

While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", I extract from your statement here that you want us to move forward to a true AI no matter what. But then...

> ...and that there is much more practical work to be done to advance humanity.

Right now AI looks like snake oil selling, you realize that, right? I stand fully behind the statement that we have tons of practical and increasingly urgent issues to solve here in the material Earth.

Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work. IMO it isn't. It's better than the medieval alchemy, but not by much.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#196

Post by pdimitar »

gbersac wrote: post #182
j2kun wrote: post #24

Earlier quoted context omitted.

I'm working on a book for programmers who want to learn math. I can send you the first few chapters if you're interested, but I'm also interested to hear your thoughts about math in general.

If you are doing this, this is what I'd love : a book with theory and ton of exercises. My usual workflow is : I read theory, kind of understand something. Then first try one type of exercise that applied that theory. I don't understand a thing. Then I read the solution and I get it. I then need at least two if not more exercises more to play with that theory before I REALLY get it in my mind. So please, add at least…

Seconded 1000 times. We have tons of books written in a pompous manner with self-patting on the back and full of theory only.

If you do indeed finish that book, consider me sold on it even if it costs $150 -- if it contains a lot of practical examples and exercises.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#197

Post by unityByFreedom »

pdimitar wrote: post #195
unityByFreedom wrote: post #168

Earlier quoted context omitted.

> the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Really disagree here. All it takes is a few well positioned people, like those at Nnaisense, to create companies promising AGI and sucking up investment dollars. We all know greed is rampant in finance, and AI tech is a big word in startups/innovation these days that can land big VC dol…

I see a small controversy in your statements, please help me understand: > Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives. While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", I extract from your sta…

> While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions",

Warnings can be faked. Fear mongering can be lessened when people are informed on a subject.

> I extract from your statement here that you want us to move forward to a true AI no matter what.

Where in the devil did you get that? I said no such thing. I think investments towards building AGI are largely a waste of money. If people want to spend their money on that, that's fine. I won't.

> Right now AI looks like snake oil selling, you realize that, right?

AGI does, yes. And, for those who do not understand the difference between data science and AGI, then probably all "AI tech" seems like snake oil. But it isn't. It's already helping doctors detect cancer, among other major issues.

> Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work

I'm not. I have no idea where you got that from my comments.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#198

Post by evolve2k »

MacsHeadroom wrote: post #187
eli_gottlieb wrote: post #162

Earlier quoted context omitted.

> This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision. That's a new area of research, actually.

New areas of research become "well studied" in a year or two in AI. GANs are considered both new and well studied, for example. Machine Learning works on a different timescale from everything else.

> Machine Learning works on a different timescale from everything else.

To piggyback your comment. I think this is the real major cause for concern, AI disruption occurs at a expotentially high rate than human learn rates affecting issues like career transition.

No one may have been ill intentioned when they applied AI and invented a way to replace anyone any longer having to mnqualy do job X but none-the-less all those who do job X are now stuck and even if they retrain (optomistixally in 2-3 years) it will in all likelyhood distrupt them again.

As technologists maybe we face a bias as we have not been significantly career disrupted by technological progress and so we can't fully see what it's like for those not riding but being swamped by the wave.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#199

Post by jacquesm »

afthonos wrote: post #93
jacquesm wrote: post #41

Earlier quoted context omitted.

> I would argue this person has no business working with AI with this sort of myopic thinking. What kind of thinking would you suggest would permit someone to 'work with AI'? Should they be quaking in their boots before being allowed to work at the altar? You could argue (and with substantially more basis in fact) that Twitter and Facebook have elevated each individuals utterings to broadcast status and that even wit…

I would challenge you to power off Amazon's or Google's infrastructure. Because a dangerous AI won't live in a desktop; it will live in a geographically replicated system, across multiple power grids and legal jurisdictions—all of which is possible today . And all these systems have spent untold hours of very smart people's time to ensure they never turn off.

That doesn't mean they don't have off switches and it does not mean they can't be turned off. I guarantee you the fire department knows how to switch any and all of those off in such a way that the diesels and the no-break PSU's all don't matter one little bit.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#200

Post by pdimitar »

unityByFreedom wrote: post #197
pdimitar wrote: post #195

Earlier quoted context omitted.

I see a small controversy in your statements, please help me understand: > Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives. While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", I extract from your sta…

> While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", Warnings can be faked. Fear mongering can be lessened when people are informed on a subject. > I extract from your statement here that you want us to move forward to a true AI no matter what. Where in the devil did you get that? I said no such thing. I think investments towards building AGI are largel…

Welp, my mistake. Might have been me triggered since the term "AI" is practically meaningless nowadays.
Post reply on HN