Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft is investing $1B in OpenAI

openai.com

Post reply on HN
Print view View original

421–430 of 623 posts

Re: Microsoft is investing $1B in OpenAI

#421

Post by chrshawkes »

andreilys wrote: post #56
kuzehanka wrote: post #17

Earlier quoted context omitted.

Is all this talk of AGI some kind of marketing meme that you guys are tolerating? We haven't figured out sentiment analysis or convnets resilient to single pixel attacks, and here is a page talking about the god damned singularity. As an industry, we've already burned through a bunch of buzzwords that are now meaningless marketing-speak. 'ML', 'AI', 'NLP', 'cognitive computing'. Are we going for broke and adding AGI…

At what point would you deem it a good idea to start working on AGI safety? What "threshold" would you want to cross before you think its socially acceptable to put resources behind ensuring that humanity doesn't wipe itself out? The tricky thing with all of this is we have no idea what an appropriate timeline looks like. We might be 10 years away from the singularity, 1000 years, or it might never ever happen! There…

I certainly wouldn't have that threshold decided by a for profit company disguised as an open source initiative to protect the world! Brought to us by Silicon Valley darlings, no thank you to that. They need to change their name or their mission. One has to go.

Re: Microsoft is investing $1B in OpenAI

#422

Post by DuskStar »

umvi wrote: post #414
DuskStar wrote: post #389

Earlier quoted context omitted.

> No one is anywhere close to achieving true AGI No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were. EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there. 0: https://intelligence.org/2017/10/13/fire-alarm/

Really? I thought by 1940 physicists generally understood fission and theoretically understood how to build a bomb - they just needed to find enough distilled fissile material (which was hard to do). And indeed, once they had enough U235, they had such a high degree of confidence in the theory, that they built a functioning U235 bomb without ever having previously tested one.

In 1939, Enrico Fermi expressed 90% confidence [0] that creating a self-sustaining nuclear reaction with Uranium was impossible. And, if you're working with U238, it basically is! But it turns out that it's possible to separate out U235 in sufficient quantities to use that instead.

On the 2nd of December, 1942 he led an experiment at Chicago Pile 1 [1] that initiated the first self-sustaining nuclear reaction. And it was made with Uranium.

In fairness to Fermi, nuclear fission was discovered in 1938 [2] and published in early 1939.

0: https://books.google.com/books?id=aSgFMMNQ6G4C&pg=PA813&lpg=...

1: https://en.wikipedia.org/wiki/Chicago_Pile-1

2: https://en.wikipedia.org/wiki/Nuclear_fission#Discovery_of_n...

Re: Microsoft is investing $1B in OpenAI

#423

Post by dragonwriter »

gonehome wrote: post #413
dragonwriter wrote: post #404

Earlier quoted context omitted.

> If AGI is achievable It almost certainly is. Humans make new intelligences all the time. > and achieving it will have consequences that dwarf everything else It probably won't, humans make new intelligences all the time. Hanging the technology base for that doesn't have any significant necessary consequences. A revolution in our ability to understand and control other intelligences might have consequences that dwar…

The difference is in the scale of the intelligence, not just the technology. It's not so much a new human like intelligence that runs on silicon, it's a general problem solving intelligence that can run a billion times faster than any individual human. This is the part I think you're underestimating. If you have that without the ability to align its goals to human goals then that's a problem.

> The difference is in the scale of the intelligence, not just the technology.

AGI is inherently no greater in scale than human intelligene, so scale is not a difference with AGI, though it might be with AGsuperI. But that's a different issue than mere AGI, and may be impossible or impractical even if AGI is doable; we have examples of human-level intelligence so we know it is physically acheivable in our universe, we don't have such examples for arbitrarily capable superhuman intelligence.

Re: Microsoft is investing $1B in OpenAI

#424

Post by Veedrac »

feral wrote: post #21

I want to take everything OpenAI says at face value (seem like good folk), but I can't help but wonder at the recent choice to keep GPT2 closed, on what seemed like pretty thin safety arguments to me. Now, the demonstrated ability to produce new models which are closed, but maybe can be used as services on a preferred partner's cloud, looks very commercially relevant? How will these conflicts be managed, or is it mor…

https://openai.com/blog/openai-lp/

Re: Microsoft is investing $1B in OpenAI

#425

Post by chrshawkes »

applecrazy wrote: post #51
gdb wrote: post #32

Earlier quoted context omitted.

Thank you!

The research that OpenAI’s doing is groundbreaking and the results are often beyond state-of-the-art. I aim to work in one of your research teams sometime!

Watch the Kool-Aid intake and you'll be just fine. Dreams are great and an absolute necessity for success but create your own. Don't buy into everything you hear, especially Elon Musk talking about Artificial General Intelligence.

Re: Microsoft is investing $1B in OpenAI

#426

Post by Quarrelsome »

mritchie712 wrote: post #420
Quarrelsome wrote: post #412

Earlier quoted context omitted.

We're making the process more opaque. How can that scale to AGI? We'll be stuck at 80% done for much longer. I would posit that while its possible, it will take so long on this tech stack that we'll find another in the interim that will produce better results. I'm not convinced this branch is the winner.

Oh, do you mean how can Azure scale to AGI? I have no opinion on Azure, I just meant someone smart will figure it out. There are huge financial incentives to do so, when that happens, we (humans) figure shit out.

> Oh, do you mean how can Azure scale to AGI?

No, not in the slightest. I mean as we progress the dev cycles get harder and slower. Then we need more engineers and the administration of more engineers working together makes everything harder.

Have you ever considered that making a rock think might be one of the greatest engineering projects our species has ever taken on? Sure humans might figure it out but I'm of the belief it will take them a very long time to. In addition, I believe that in that timescale a different tech stack might show more promise. I'm not convinced this technological branch scales all the way to AGI.

Re: Microsoft is investing $1B in OpenAI

#427

Post by [deleted] »

chirau wrote: post #314
pinouchon wrote: post #281

What do they mean by: "Microsoft will become our exclusive cloud provider"? Being forced to use Azure for all your ML workloads seems a stupid constraint. For example, you might be comfortable with tensorflow/TPU and changing frameworks/tooling might be costly.

Azure has full support for Tensorflow, Keras, PyTorch and the rest of the popular stuff. Shouldn't be a problem at all

[deleted]

Re: Microsoft is investing $1B in OpenAI

#428

Post by Veedrac »

Alas1 wrote: post #289
sp332 wrote: post #145

Earlier quoted context omitted.

Sentiment analysis is at 96% and increasing rapidly. http://nlpprogress.com/english/sentiment_analysis.html

What exactly does that 96% mean, though? It means that on some fixed dataset you're achieving 96% accuracy. I'm baffled by this stupidity of claiming results (even high-profile researchers do this) based on datasets with models that are nowhere near as robust as the actual intelligence that we take as reference: humans. Take the model that makes you think "sentiment analysis is at 96%", come up with your own examples…

> I think continual Turing testing is the only way of concluding whether an agent exhibits intelligence or not.

So you think it's impossible to ever determine that a chimpanzee, or even a feral child, exhibits intelligence? This seems rather defeatist.

Re: Microsoft is investing $1B in OpenAI

#429

Post by streetcat1 »

z3t4 wrote: post #385
marvin wrote: post #349

Earlier quoted context omitted.

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a prag…

The atomic bomb was based on science theory. A computer can run many programs and do a great many things, but it will never be able to think by itself.

It does not need to. It just need to get complex enough. This is from an 1965 article:

"If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines’ decisions. As society and the problems that face it become more and more complex and as machines become more and more intelligent, people will let machines make more and more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machine off, because they will be so dependent on them that turning them off would amount to suicide."

Re: Microsoft is investing $1B in OpenAI

#430

Post by chrshawkes »

breck wrote: post #227
pron wrote: post #211

Earlier quoted context omitted.

I was actively engaged in the field in the late-nineties when AI was also five years around the corner. I’ve mostly lost interest since then, and the disappointment that is deep learning has only dulled my enthusiasm further (not that it hasn’t achieved some cool things, but it’s a long way from where we’d thought we’d be by now).

> I was actively engaged in the field in the late-nineties So was Geoffrey Everest Hinton > I’ve mostly lost interest since then but he didn't give up. If you expect someone to just hand us AGI in a nicely wrapped package with a bow, with all the details neatly described, you are absolutely right, that's really far off! But for the record there are many people actively grinding it out in the field, day in and day out…

He's also telling us we're going in the wrong direction and have been with our approach to reinforced learning. He's not convinced that's how the brain works. In fact he's convinced it's not.
Post reply on HN