Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft is investing $1B in OpenAI

openai.com

Post reply on HN
Print view View original

381–390 of 623 posts

Re: Microsoft is investing $1B in OpenAI

#381

Post by pnathan »

013a wrote: post #279
gdb wrote: post #76

Earlier quoted context omitted.

(I work at OpenAI.) It comes down to whether you believe AGI is achievable. We've talked about why we think it might be: https://medium.com/syncedreview/openai-founder-short-term-ag... , https://www.youtube.com/watch?v=YHCSNsLKHfM And we certainly have more of a plan for building it than warp drives :). EDIT: I personally think the case for near-term AGI is strong enough that it'd be hard for me work on any other pro…

"Guardrails" is such a cute little term. AGI is a twenty-ton semi filled with rocket fuel. Guardrails won't stop it from careening into an elementary school if it decides that's its most optimal course of action. Mostly because, despite the previous analogy; no one knows what AGI is. No one knows what it will look like. No one will know when we've created it. No one even knows what INTELLIGENCE is in humans. How can…

Eventually a toddler gets over the guardrails, except for the ones which are materially disabled.

I have deep and profound doubts about the notion of guardrails for general intelligence; without even considering the ethical concerns, a general intelligence should be able to simply rewire itself to achieve what it wants. a key part of self-reflection and learning is that rewiring.

so I think that it's a self-defeating notion on the face of it.

(note that I do not have comment on the actual dangers involved here, but only on the philosophy)

Re: Microsoft is investing $1B in OpenAI

#382

Post by nradov »

marvin wrote: post #349
DSingularity wrote: post #313

Earlier quoted context omitted.

Eventually open ai?

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a prag…

Not to worry. No one is anywhere close to achieving true AGI so any safety concerns are a moot issue. It's akin to worrying about an alien invasion.

Re: Microsoft is investing $1B in OpenAI

#383

Post by bogidon »

jorjordandan wrote: post #308
bogidon wrote: post #257

Earlier quoted context omitted.

Pron, I fully support your take here. Most AGI campaigners here clearly think that we must have already figured out a lot about how consciousness works. But is there any evidence to back that up? No, because we _haven't_ created consciousness. The most we've done is manipulate _existing_ consciousness. Sure, we can point to similarities between deep learning and the brain, and these avenues are interesting and, I thi…

Consciousness may not have anything to do with AGI. Besides, we haven't as a species defined consciousness in a consistent and coherent way. It may be an illusion or a word game. AGI may end up being more like evolution, a non-conscious self optimizing process. Everyone is talking about AGI but we can't even define what we mean by any of these terms, so to put limits on how near or far away major discoveries might be…

True. I used “consciousness” haphazardly.

Not super related, but AGI enthusiasts sometimes remind me of this: https://youtu.be/bS5P_LAqiVg?t=9m50s

Re: Microsoft is investing $1B in OpenAI

#384

Post by TheIronYuppie »

mr_overalls wrote: post #312
TheIronYuppie wrote: post #221

Earlier quoted context omitted.

Disclosure: I work at Azure in AI/ML Another way to think about it is that for folks building in .NET, ML.NET makes it easy for them to start using many of the ML techniques without having to learn something new. On top of that, we FULLY support all the industry standard tools - TF, Keras, PyTorch, Kubeflow, SciKit, etc etc. We even ship a VM that lets you get started with any of them in one click ( https://azure.mic…

Nice, thanks for the info. Maybe it's proper to think of ML.NET as training wheels, then!

For what it's worth, we are pretty proud of the performance as well - I wouldn't call it training wheels :)

On both scale up and run times, it measures up as among the best-in-class[1]. That is to say, for the scenarios which people use it most commonly (GBT, linear learners), it's a great fit!

[1] https://arxiv.org/pdf/1905.05715.pdf

Re: Microsoft is investing $1B in OpenAI

#385

Post by z3t4 »

marvin wrote: post #349
DSingularity wrote: post #313

Earlier quoted context omitted.

Eventually open ai?

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a prag…

The atomic bomb was based on science theory. A computer can run many programs and do a great many things, but it will never be able to think by itself.

Re: Microsoft is investing $1B in OpenAI

#386

Post by ryanmercer »

pron wrote: post #369
ryanmercer wrote: post #317

Earlier quoted context omitted.

>I can't tell if you're serious, but assuming you are, I assume he is, given he is Greg Brockman the CTO and a co-founder. I know Sam Altman is similarly optimistic, having told me on multiple occasions something along the lines of 'I can't focus on anything else right now' which in context I very much took as 'this presently consumes my waking thoughts and I only have time for it'. This sort of drive is great, but I…

https://www.buzzfeednews.com/article/tedchiang/the-real-dang...

Exactly, and if they do create an AGI it's probably going to be a lot like the creators as its original parameters were set by them.

Just look at Amazon's warehouse algorithm:

~Biotic unit achieved goal, raise goal~

~Biotic unit achieved new goal, raise goal~

~Biotic unit achieved new new goal, raise goal~

~Biotic unit failed new new goal, replace biotic unit~

~New biotic unit failed new new goal, replace biotic unit~

~New new biotic unit failed new new goal, replace biotic unit~

With Amazon though, a human can eventually go "wow, w're firing new hires within the first 3 weeks like 97% of the time, and 100% within 6 weeks, erm, let's look at this algorithm".

But if you create an AGI that has the Silicon Valley mindset "we will do this, because we have to do this" (an exact quote I heard from an individual while in the Bay Area to stop by OpenAI : "We will figure out global warming, because we have to") then the AGI is probably going to be designed with the 'mindset' that "Failure is not an option, a solution exists, continue until solution is found" which, uh, could be really bad depending on the problem.

Here's a worst case scenario:

"I am now asking the computer how to solve climate change"

~~beep boop beep boop, beep beep, boooooooop~~ the CO2 emissions are coming from these population centers.

~~boop beep beep beep boop boop boop boop beep boop~~ nuclear winter is defined as: a period of abnormal cold and darkness predicted to follow a nuclear war, caused by a layer of smoke and dust in the atmosphere blocking the sun's rays.

~~boop beep beep, boop~~ Project Plowshare and Nuclear Explosions for the National Economy were projects where the two leading human factions attempted to use nuclear weaponry to extinguish fires releasing excessive carbon dioxide as well as for geoengineering projects. Parameters set, nuclear weapons authorized as non-violent tools.

~~beep beep beep boop boop boop, beep boop, beep, boop, beep~~ I now have control of 93% of known nuclear weapons, killing the process of 987 of the most populous cities will result in sufficient reduction for the other biotic species to begin sequestering more carbon than is produced, fires caused by these detonations should be minimal and smaller yield weapons used as airbursts should be capable of extinguishing them before they can spread. Solution ready. Launching program.

Watch officer at NORAD some time later "Shit, who's launching our nuke?!?"

Someone else at NORAD "they're targeting our own major population centers!"

Somewhere in Russia "Our nuclear weapons are targeting our own cities!"

Somewhere in Pakistan "our nuclear weapons are targeting our own cities!"

somewhere...

Re: Microsoft is investing $1B in OpenAI

#387

Post by Jach »

intrasight wrote: post #325
Jach wrote: post #252

Earlier quoted context omitted.

You piqued my interest, what makes you think there's any intelligence, artificial or otherwise, anywhere else but here in the Milky Way?

Space is big and time is long. You may have been thrown by a typo in my comment. Was "I've do doubt" but meant "I've no doubt"

That's precisely why I don't believe there is any interesting intelligent life out there, at least in our neck of the woods (the Milky Way). Mostly the time aspect. The galaxy is big, but not so big that it would take more than 200 million years to send a von-neumann probe to every star. The galaxy itself is only slightly younger than the universe, over 10 billion years. In all that time not a single intelligence has arisen and advanced just a bit beyond our level to at least start a mapping project, let alone anything like megascale structures and other projects we would expect instrumental convergence on? That only makes sense to me if we live in a mostly dead galaxy that has always been mostly dead (whether the filter is ahead or behind us makes little difference apart from judging our own prospects), or if we're so deluded about the nature of reality that we might as well say God has tricked us into thinking we're alone until we're spiritually ready or whatever.

Re: Microsoft is investing $1B in OpenAI

#388

Post by Quarrelsome »

noir_lord wrote: post #360
benl wrote: post #346

If you strip out the AGI hype then this just sounds like OpenAI is now moving to monetizing their tech. This makes sense for them but probably not for the philanthropists who originally backed them. Sadly for them, AGI is metaphysically impossible - this will be realized eventually but a lot of waste and possibly harm will happen first. We are not just super sophisticated machines, so the fact that we can think doesn…

> But philosophy does - and it tells us you can’t get mind from matter, no matter what configuration you put it in. Curious - do you think humans have mind? because if so we are very much matter and if not well that's an interesting thought as well.

A framing: can we make a rock think?

I don't know the answer but that some people think they do upsets me. I definitely think we should try but right now mostly what we do is make a rock DO so I'm not seeing the leap yet.

Re: Microsoft is investing $1B in OpenAI

#389

Post by DuskStar »

nradov wrote: post #382
marvin wrote: post #349

Earlier quoted context omitted.

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity. Glad to see that the team is taking a prag…

Not to worry. No one is anywhere close to achieving true AGI so any safety concerns are a moot issue. It's akin to worrying about an alien invasion.

> No one is anywhere close to achieving true AGI

No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were.

EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there.

0: https://intelligence.org/2017/10/13/fire-alarm/

Re: Microsoft is investing $1B in OpenAI

#390

Post by mathgenius »

noir_lord wrote: post #360
benl wrote: post #346

If you strip out the AGI hype then this just sounds like OpenAI is now moving to monetizing their tech. This makes sense for them but probably not for the philanthropists who originally backed them. Sadly for them, AGI is metaphysically impossible - this will be realized eventually but a lot of waste and possibly harm will happen first. We are not just super sophisticated machines, so the fact that we can think doesn…

> But philosophy does - and it tells us you can’t get mind from matter, no matter what configuration you put it in. Curious - do you think humans have mind? because if so we are very much matter and if not well that's an interesting thought as well.

But, not even matter is "very much matter".

I'm a quantum maximalist: the brain is just the antennae, receiving and broadcasting. Attention itself cuts (slices) through the quantum soup, and as a result, these mind-forms appear.

Post reply on HN