Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft is investing $1B in OpenAI

openai.com

Post reply on HN
Print view View original

271–280 of 623 posts

Re: Microsoft is investing $1B in OpenAI

#271

Post by apta »

gonehome wrote: post #237
pron wrote: post #80

Earlier quoted context omitted.

> It comes down to whether you believe AGI is achievable. No, it does not. I very much believe AI (or AGI, as you call it) is achievable, but may I remind you that some years after the invention of neural networks, Norbert Wiener, one of the greatest minds of his generations, said that the secret of intelligence would be unlocked within five years, and Alan Turing -- a component of your very own post-pre-AGI era's AG…

If AGI is achievable (seems likely given brains are all over the place in nature) and achieving it will have consequences that dwarf everything else then doesn't it make sense to focus on it? Yes, historically people were way too optimistic and generally went down AI rabbit holes that went nowhere, but two years before the Wright flyer flew, the Wright brothers themselves said it was 50 years out (and others were sti…

> If AGI is achievable (seems likely given brains are all over the place in nature)

I don't see how that conclusion follows the antecedent.

Re: Microsoft is investing $1B in OpenAI

#272

Post by houzertuch »

richardwhiuk wrote: post #255
houzertuch wrote: post #244

I don’t understand how there can be no comments regarding the fatal flaw of AGI which is that it will completely ruin the economics of the world. The world is the way it is now because humans are the only source of intelligent signal processing. That’s the only reason why humans enjoy the limited rights and privileges that they do. That’s the only reason why life has gotten better and better with advancing medicine a…

Is your alternative "Don't invent general AI?"? If so that seems unbelievably naive. Things generally can't be un-invented, and it's unimaginably hard to prevent people inventing things, especially with such a large economic upside for inventing it.

Prove me wrong if it is so obvious.

As the people at OpenAI have rightly said, AGI is a compute-gated problem. It is a problem that can only be solved with very, very large amounts of compute.

The world has some total amount of computing power in terms of silicon based computation. For AGI to happen, there are two requirements: that this total be equal or greater than some theoretical threshold value for AGI and that the computing power is consolidated. So in layman’s terms, you have to have a lot of computers and they have to be connected in such a way as to efficiently share their compute. AGI will never come about if every individual computer were used to do research by separate entities but if all of those computers were connected into a single virtual computer, AGI might be discovered with them.

So clearly in order to prevent AGI, the best thing to do would be to address these two aspects. Prevent the total computing power of the world from growing and prevent computers from forming virtual meta-computers. Both of these tasks are in principle extremely easy.

Chip fabs are huge and expensive. Nobody is fabricating chips in their garage. This is just a hard fact. There aren’t that many fabs in the world and they are all highly susceptible to regulation. This isn’t prohibition of alcohol so please don’t confuse yourself. Nobody will be brewing chips in their cellar.

Let’s imagine that you could not regulate cloud computing. Let’s say the only way to prevent computers from offering their compute on a virtual market was to shut down the internet. This by default is the hardest way to solve the second aspect of the AGI problem and even it is very easy. This is because the internet is a large fragile collection of infrastructure that depends heavily on government cooperation. Nobody is going to string a fiber backbone for black market internet. ISPs cannot exist without regulatory approval.

If there were political awareness and motivation, and it was a global phenomenon, yes, it would be extremely easy to do what I’ve described. And since AGI is to the detriment of literally all people, it is not a far fetched scenario. And unlike alcohol in the United States, bootlegging would not be a problem. People in the USA think that banning anything whatsoever doesn’t work. It’s just fuzzy thinking, I can assure you.

Re: Microsoft is investing $1B in OpenAI

#273

Post by dr_dshiv »

pron wrote: post #228
breck wrote: post #219

Earlier quoted context omitted.

> the topic has been subject to overoptimism for decades now, But so has every other big idea that went on to become reality, like planes (da Vinci was drawing designs for planes over 400 years before the first working ones). > no one knows whether what they're doing is even on the right path towards AI This is completely wrong. That would be like saying "no one knows if working on a wing is on the right path to flig…

> But so has every other big idea that went on to become reality, like planes (da Vinci was drawing designs for planes over 400 years before the first working ones). And so has every other big idea that didn't become reality, and that was the majority. Again, I have no problem with AI research whatsoever, but the prose was still eyebrow-raising considering the actual state of affairs. > They are very similar. They ar…

Do you have a reference on the brain not learning with backpropagation? I'd like to learn more.

Re: Microsoft is investing $1B in OpenAI

#274

Post by gonehome »

Quarrelsome wrote: post #258
mritchie712 wrote: post #249

Earlier quoted context omitted.

This is especially true considering we're talking about software vs. hardware (airplane). A few, or even one, brilliant mind(s) could make a break thru in AGI in a matter of months.

> A few, or even one, brilliant mind(s) could make a break thru in AGI in a matter of months. same goes for warp drives doesn't it? The point is that people don't see how we build THAT out of these tools we currently have. We only build pastiches of intelligence today and either we have an arrogant view of the level of our own intelligence or we can't make THAT with THIS. But maybe warp-drives, maybe world-peace too?

Not really - we're not sure if warp drive is possible given the physical constraints of the universe.

AGI is possible because intelligence is possible (and common on earth) in nature.

Re: Microsoft is investing $1B in OpenAI

#275

Post by breck »

igorkraw wrote: post #242
breck wrote: post #194

Earlier quoted context omitted.

> calling it "pre-AGI" pretentious to the level of delusion. I don't think you know what you are talking about. Do you do Deep Learning? If you are not actively engaged in the field, I wouldn't be so quick to dismiss others who are (especially not others who are at the top of the field). That being said, you brought up some interesting points, even if I think your overall position is wrong--I think OpenAI is definite…

> > calling it "pre-AGI" pretentious to the level of delusion. >I don't think you know what you are talking about. Do you do Deep Learning? If you are not actively engaged in the field, I wouldn't be so quick to dismiss others who are (especially not others who are at the top of the field). I do (or at least I try, I get money for my attempts though) and I concur with calling it delusion. So does Francois Cholet. So…

> the less-than-perfectly aligned government etc. will strongarm you into giving control to THEM before you come close to deploying it

and

> because you think you are smart enough to build AGI without these elites finding out AND smart and/or wise enough to do what is best for humanity

Are very good points and I share those concerns too, and have no good answers. I'm in the pessimist camp when it comes to AGI--I would be heavily it's going to happen but I wouldn't bet a dollar whether it will end up being good for humanity, as I haven't a clue.

Re: Microsoft is investing $1B in OpenAI

#276

Post by epiphanitus »

houzertuch wrote: post #215
gdb wrote: post #76

Earlier quoted context omitted.

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/openai-founder-short-term-ag... , https://www.youtube.com/watch?v=YHCSNsLKHfM And we certainly have more of a plan for building it than warp drives :). EDIT: I personally think the case for near-term AGI is strong enough that it'd be hard for me work on any other pro…

AGI will the the worst thing that has ever happened to humanity. Even if you can’t see that, you can see that it has the potential to be very bad. I know this because OpenAI literature always stresses that AGI has to be guided and developed by the right people because the alternative is rather unpleasant. So essentially its a gamble and you know it. With everyone’s lives at stake. But instead of asking everyone wheth…

I'm curious as to why you think AGI will have an inherently bad effect on society. Personally, I have a hard time believing any technology is inherently good or bad.

On the other hand, if a society's institutions are weak, it's leaders evil or incompetent, or its people are uneducated, its not hard to imagine things going very, very wrong.

Re: Microsoft is investing $1B in OpenAI

#277

Post by j88439h84 »

breck wrote: post #210
pron wrote: post #193

Earlier quoted context omitted.

> They are making breakthroughs/incremental advances that required to get there They don't know that. We have no idea what's required to achieve AI. Now I don't know how long before Ford actually built cars he started saying he's building cars, but if Wikipedia is to be believed, it could not have been more than three-four years. Also, when he started building cars, he pretty much knew what's required to build them.…

> We have no idea what's required to achieve AI Yes, we do. Lots of data, lots of training, better algorithms, more understanding of the brain...At this point we still need 10x+ improvements in a lot of areas, but it's pretty clear what we need to do. If you can process around 100 petabytes per second (1 Google Index of data per second), you could fully simulate a human being, including their brain. We're still a lit…

> doing deep learning, and do research on programming languages and deep learning for program synthesis.

That sounds fascinating. Could you link to some relevant stuff about languages and deep learning for program synthesis? I'd love to read more about this.

Re: Microsoft is investing $1B in OpenAI

#278

Post by joe_the_user »

pron wrote: post #254
gonehome wrote: post #237

Earlier quoted context omitted.

If AGI is achievable (seems likely given brains are all over the place in nature) and achieving it will have consequences that dwarf everything else then doesn't it make sense to focus on it? Yes, historically people were way too optimistic and generally went down AI rabbit holes that went nowhere, but two years before the Wright flyer flew, the Wright brothers themselves said it was 50 years out (and others were sti…

First of all, I was talking the language, not the work. It makes sense to study AI as it does many other subjects, but we don't know that it will "have consequences that dwarf everything else" because we don't know what it will be able to do and when (we think that it could but so could, say, a supervirus, or climate change, or the return of fascism). People hang all sort of dreams on AI precisely because of that. Th…

We don't know whether AGI is possible or even exactly what it is. However, if a form of intelligence where adding more hardware adds more capabilities in the fashion of present computing but where the capacities are robust and general purpose like humans rather than fragile and specialized like current software, then we'd have of amazing power - brilliant people can amazing things. A device that's akin to an army of well-organized brilliant people in a box clearly would many capacities. So it's reasonable to say that if that's possible, investing in it may have a huge payoff. (Edit: the "strong" "AGI is possible" would be that AGI is an algorithm that gives a computer human-like generality and robustness while having ordinary soft-like-abilities. There are other ideas of AGI, of course - say, a scheme that would simulate a person on such a high level that the simulated person had no access to the qualities of the software doing the simulation but that's different).

The problem, however, I think another gp's objection. OpenAI isn't really working on AGI, it's making incremental improvements on tech that's still fragile and specialized (maybe even more specialized and fragile), where the only advance of neural nets is that now they can be brute-force programmed.

Re: Microsoft is investing $1B in OpenAI

#279

Post by 013a »

gdb wrote: post #76
pron wrote: post #40

Congrats on the investment, but this release reads like a parody. I believe that building a beneficial warp-drive engine will be the most important technological development in human history, with the potential to shape the trajectory of humanity. The aliens we're sure to encounter will be capable of mastering more fields than any one human — like a tool which combines the skills of Curie, Turing, and Bach. An alien…

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/openai-founder-short-term-ag... , https://www.youtube.com/watch?v=YHCSNsLKHfM And we certainly have more of a plan for building it than warp drives :). EDIT: I personally think the case for near-term AGI is strong enough that it'd be hard for me work on any other pro…

"Guardrails" is such a cute little term. AGI is a twenty-ton semi filled with rocket fuel. Guardrails won't stop it from careening into an elementary school if it decides that's its most optimal course of action. Mostly because, despite the previous analogy; no one knows what AGI is. No one knows what it will look like. No one will know when we've created it. No one even knows what INTELLIGENCE is in humans.

How can you create effective guardrails when you have no concrete idea what the vehicle you're trying to stop is? Turns out, AGI comes along, and it's an airplane. Great guardrails, buddy.

And, you know, lets go a step further; you've got great guardrails in-place here in beautiful, free America. Against all odds, they work. Then, China or Russia pay one of your employees $250M to steal the secret. Or, they develop it independently. Are they going to use the guardrails, or will they "forget" to include that flag? A disgruntled employee leaks it to the dark web, and now everyone has it. I don't even wear a helmet when I'm riding a bike. How the hell can you expect this technology to be anything but destructive?

The only path forward is to speak with a single voice to the governments of the world, that we need to Stop This. AGI research should be subject to the same sanctions that nuclear weapons development is. You communicate with quips and cute emojis like none of what you're doing matters, but AGI easily ranks among the top three most likely ways we're going to Kill Our Species. Not global warming; we'll survive that. Not a big meteor strike; thats rare and predictable. But the work you're doing right now.

Re: Microsoft is investing $1B in OpenAI

#280

Post by gonehome »

mikorym wrote: post #262
gonehome wrote: post #237

Earlier quoted context omitted.

If AGI is achievable (seems likely given brains are all over the place in nature) and achieving it will have consequences that dwarf everything else then doesn't it make sense to focus on it? Yes, historically people were way too optimistic and generally went down AI rabbit holes that went nowhere, but two years before the Wright flyer flew, the Wright brothers themselves said it was 50 years out (and others were sti…

Why would AGI dwarf anything? There are at least 7 billion beings on the planet with AGI already. I think a bigger problem is the general well being of the aforementioned 7 billion entities.

For a simple thought experiment take one human brain architecture (say it can operate at 100 operations per second), scale that up to a billion operations per second. It can think centuries worth of human thinking in a couple of hours.

If you have an AGI that has goals that are not aligned with your interests it'll dwarf everything else because it thinks faster than you (and can therefore act faster than you) in pursuit of its goals.

Post reply on HN