Microsoft is investing $1B in OpenAI • Hacker News Board

Re: Microsoft is investing $1B in OpenAI

#371

Post by Veedrac » Mon, Jul 22, 2019, 7:38 PM UTC

craigsmansion wrote: post #128

andreilys wrote: post #56

Earlier quoted context omitted.

At what point would you deem it a good idea to start working on AGI safety? What "threshold" would you want to cross before you think its socially acceptable to put resources behind ensuring that humanity doesn't wipe itself out? The tricky thing with all of this is we have no idea what an appropriate timeline looks like. We might be 10 years away from the singularity, 1000 years, or it might never ever happen! There…

> At what point would you deem it a good idea to start working on AGI safety? I don't know, but I'd say after a definition of "AGI" has been accepted that can be falsified against, and actually turn it into a scientific endeavour. > The tricky thing with all of this is we have no idea what an appropriate timeline looks like. We do. As things are it's undetermined, since we don't even know what's it's supposed to mean…

> I don't know, but I'd say after a definition of "AGI" has been accepted that can be falsified against, and actually turn it into a scientific endeavour.

Uhh, that's the Turing Test.

Re: Microsoft is investing $1B in OpenAI

#372

Post by benl » Mon, Jul 22, 2019, 7:40 PM UTC

mitchellgoffpc wrote: post #348

benl wrote: post #346

If you strip out the AGI hype then this just sounds like OpenAI is now moving to monetizing their tech. This makes sense for them but probably not for the philanthropists who originally backed them. Sadly for them, AGI is metaphysically impossible - this will be realized eventually but a lot of waste and possibly harm will happen first. We are not just super sophisticated machines, so the fact that we can think doesn…

I would ask for evidence to support your claim, but I think Newton’s Flaming Laser Sword probably applies in this case.

Yes, I used to hold that view too. But actually it turns out that the null hypothesis is that mind is at least partly immaterial, because all attempts to demonstrate the opposite philosophically are fraught with difficulty. I’ve found that the thought of Aristotle and Aquinas, when explained by modern philosophers, best explains to me why that’s the case.

Re: Microsoft is investing $1B in OpenAI

#373

Post by JamesBarney » Mon, Jul 22, 2019, 7:41 PM UTC

pron wrote: post #351

JamesBarney wrote: post #204

Earlier quoted context omitted.

I think you're issue with the language is not shared by most people. In research we rarely know beforehand what research is on the right or wrong path. But we are comfortable with someone saying they are researching something even if they don't know beforehand whether the research will be useful or a wild goose chase. For example most people's first though to hearing "I'm researching ways to treat Alzheimer's" isn't…

Yeah, in this release they're not saying they're doing research towards AI, or even that they're researching AI. They're saying that they're " building artificial general intelligence" and developing a platform that " will scale to AGI." (emphasis mine) They're also calling what they're actually building "pre-AGI."

> We’re partnering to develop a hardware and software platform within Microsoft Azure which will scale to AGI.

This sentence might by itself imply they are farther along than they are, but in the context of the whole article I never got the impression they were close to actually building an AGI.

> The most obvious way to cover costs is to build a product, but that would mean changing our focus. Instead, we intend to license some of our pre-AGI technologies, with Microsoft becoming our preferred partner for commercializing them.

This read pretty straightforwardly to me. Pre-AGI seems like a shorthand for useful technologies like GPT-2.

Reading the article I never got the impression they'd solved AGI, or were even close. The context of the article is a partnership announcement not a breakthrough. I could see how a few people who are very unsophisticated might get a little confused as to how far along they are. But I assumed they were writing for people who had heard of OpenAI which pretty much eliminates anyone this unsophisticated.

Re: Microsoft is investing $1B in OpenAI

#374

Post by gonehome » Mon, Jul 22, 2019, 7:43 PM UTC

pron wrote: post #363

gonehome wrote: post #295

Earlier quoted context omitted.

I can understand your point about the language, but I guess I think it's reasonable to set the goal for what you actually want and work towards it. It may turn out to be unattainable, but I think generally you need to at least set it as the goal. It also seems less clear to me that they are close or far from it (I don't think it's on the same level as warp drive). I don't know about the god thing you mention and the…

I think discussions of AI safety at this stage -- when we're already having problems with what passes for AI these days that we're not handling well at all -- is a bit silly, but I don't have something particularly intelligent to say on the matter, and neither, it seems, does anyone else, except maybe for this article that shows that the AGI paranoia (as opposed to the real threats from "AI" we're already facing, lik…

I agree with you in a way, if AGI ends up being 300yrs out then work on safety now is likely not that important since whatever technology is developed in that time will probably end up being critical to solving the problem.

My main issue personally is that I'm not confident if it's really far out or not and people seem bad at predicting this on both sides. Given that, it probably makes sense to start the work now since goal alignment is a hard problem and it's unknown when it'll become relevant.

I read the BuzzFeed article and I think the main issue with it is he assumes that an AGI will be goal aligned by the nature of being an AGI:

"In psychology, the term “insight” is used to describe a recognition of one’s own condition, such as when a person with mental illness is aware of their illness. More broadly, it describes the ability to recognize patterns in one’s own behavior. It’s an example of metacognition, or thinking about one’s own thinking, and it’s something most humans are capable of but animals are not. And I believe the best test of whether an AI is really engaging in human-level cognition would be for it to demonstrate insight of this kind."

Humans have general preferences and goals built in that have been selected for for thousands of years. An AGI won't have those by default. I think people often think that something intelligent will be like human intelligence, but the entire point of the strawberry example is that an intelligence with different goals that's very good at general problem solving will not have 'insight' that tells it what humans think is good (that's the reason for trying to solve the goal alignment problem - you don't get this for free).

He kind of argues for the importance of AGI goal alignment which he calls 'insight', but doesn't realize he's doing so?

The comparison to Silicon Valley being blinded by the economies of their own behavior is just weak politics that's missing the point.

Re: Microsoft is investing $1B in OpenAI

#375

Post by JamesBarney » Mon, Jul 22, 2019, 7:43 PM UTC

sgt101 wrote: post #265

JamesBarney wrote: post #204

Earlier quoted context omitted.

I think you're issue with the language is not shared by most people. In research we rarely know beforehand what research is on the right or wrong path. But we are comfortable with someone saying they are researching something even if they don't know beforehand whether the research will be useful or a wild goose chase. For example most people's first though to hearing "I'm researching ways to treat Alzheimer's" isn't…

Although, perhaps you would agree that someone saying "my work on Alzheimer's might help your friend" would be behaving in a cruel and unprofessional way unless the treatment was indeed in human trials?

I read the article more as "maybe my work will one day be able to cure Alzheimer's and help people like your friend".

What in the article gave you the impression they had made a large breakthrough or were close to an AGI?

Re: Microsoft is investing $1B in OpenAI

#376

Post by cosmodisk » Mon, Jul 22, 2019, 7:46 PM UTC

pure-awesome wrote: post #58

Quarrelsome wrote: post #44

Can we talk about the usage of the term "AGI" here? Considering its connotations in popular culture it sounds terrifically inappropriate in terms of what we can feasibly build today. Can we assume that marketing overrode engineering on the terminology of this press release?

The purpose of OpenAI is to eventually lead to safe AGI. It's part of their core business purpose. Whatever they do with Machine Learning today is merely instrumental in leading up to that goal. We certainly cannot feasibly build AGI today, hence OpenAI's use of the term "pre-AGI technologies".

I can't see how pure AGI can be "safe". Huge part of human intellect revolves around the need to survive,be it danger,lack of food,or less rational choices based on emotions.If computers can rationalise positive behaviour of humans,they may not be able to do so well with greed, jealousy, hunger for power,that aren't very logical processes but create a lot of positive and negative nonetheless.

Re: Microsoft is investing $1B in OpenAI

#377

Post by alluro2 » Mon, Jul 22, 2019, 7:48 PM UTC

MegaButts wrote: post #186

chillacy wrote: post #173

Earlier quoted context omitted.

So you’re saying.. it comes down to whether you believe AGI is achievable within our lifetime? Or even worth contributing to at all even. I think the parent has made their position pretty clear through their employment choices, that’s a level of skin in the game that naysayers don’t really have.

There are plenty of people who worked on AI (as graduate students, as ML researchers at hot startups for self-driving cars, as pure researchers or supporting engineers at Google or Facebook, and many other places), and then left because once they saw how limited the research was they lost hope it was going to happen before they died. Also while I fully understand and appreciate the necessity of OpenAI abandoning the…

I honestly don't see any possibility of truly "wide-spread benefits for humanity" if AGI is achieved anytime soon. Current state of humanity when it comes to how we treat each other and anything akin to species-level awareness and collaboration is only barely better in recent history than the dark ages. If a group of people gets access to an AGI, I think it will very quickly result in a bit wider group than that having their lives prolonged, no disease, no lack of resources and practically infinite wealth, and everyone eventually being either rid off, or allowed to live in far-away slums and left to die off.

Re: Microsoft is investing $1B in OpenAI

#378

Post by AlexCoventry » Mon, Jul 22, 2019, 7:49 PM UTC

breck wrote: post #269

pron wrote: post #263

Earlier quoted context omitted.

The kind of language used in this release has actually hurt AI considerably before, so by pointing out that it's delusional I am not giving up and helping save AI from the research winter that OpenAI seems to be working on. You're welcome, AI!

okay I'll concede your point that perhaps being bold could be bad publicity for the field. I think that's a reasonable position to take. I don't think it is correct, but I think it's reasonable. Even if it were the precursor to a drop in funding, I don't think the previous "AI Winter" was so long, in comparison to the century-long gaps in the advance of other technologies in history, (binary was invented hundreds of…

> They are simply stating what the math tells them.

Which math?

Re: Microsoft is investing $1B in OpenAI

#379

Post by the8472 » Mon, Jul 22, 2019, 7:55 PM UTC

adrianmonk wrote: post #324

tim333 wrote: post #225

Earlier quoted context omitted.

There's quite a lot of experimental evidence in the "purely physical phenomenon" direction. Things like brain surgery, neural implants and mind altering drugs probably wouldn't work as well if they were trying to interact with your non physical spirit.

I definitely see that. It's pretty compelling. You can't deny that certain drugs, for example, are a lever you can pull that makes consciousness go away (or come back). There seems to be a cause-effect relationship there. But at the same time, I do not understand how it's possible that the consciousness that I subjectively experience can arise from physical processes. Therefore, I have difficulty completely accepting…

If you do not understand what you mean yourself by the word "consciousness" then it is futile to ask whether an object has that property.

For example the purpose of anesthesia the goal can be broken down into several sub-components that you need to turn off without ever invoking the concept of consciousness: wakefulness, memory-formation, sensory input (pain).

Similarly consciousness seems to be a grab-bag of fuzzy properties that we ascribe to humans and then by letting five be even we also allow a few other species to roughly match (some) of those properties if we squint. And since humans and other, clearly somewhat simpler, species are clearly conscious then we go on and declare it's a really difficult thing to understand how ever-simpler things can't be conscious. It's just the paradox of the heap.

This doesn't mean consciousness is magical. It's just a very poorly defined and overloaded concept almost bordering on useless in the general case. It may feel like magic because we built this thought-edifice that twists to escape our grasp. But to me that seems more like a philosophical mirror cabinet that distracts from looking at the actual problem.

If you want to ask whether something is conscious you first need to come up with a rigorous testable definition or break it down into smaller components which you can detach from the overloaded concept of consciousness.

Re: Microsoft is investing $1B in OpenAI

#380

Post by houzertuch » Mon, Jul 22, 2019, 7:57 PM UTC

epiphanitus wrote: post #327

houzertuch wrote: post #244

I don’t understand how there can be no comments regarding the fatal flaw of AGI which is that it will completely ruin the economics of the world. The world is the way it is now because humans are the only source of intelligent signal processing. That’s the only reason why humans enjoy the limited rights and privileges that they do. That’s the only reason why life has gotten better and better with advancing medicine a…

>>The world is the way it is now because humans are the only source of intelligent signal processing. That’s the only reason why humans enjoy the limited rights and privileges that they do. That’s the only reason why life has gotten better and better with advancing medicine and so on. That's actually an interesting point I haven't thought of before. I disagree with you though. You could have made the same argument du…

Industrial revolution: has nothing to do with this. Some forms of human labor were automated. The set of all human abilities was not threatened. AGI is fundamentally different because AGI will automate everything a human can do. Increasingly important distinction. And not something I would overlook in the hundreds of hours of rumination I’ve put into it.

Can’t stop countries from doing it: yes, we can. A tin can dictatorship can’t do it. And besides, it’s a problem that depends a lot on academic work being published and shared among researchers. It’s a collaborative effort largely. Some rogue state trying to do this while avoiding inspections and inquiries by larger nations, and with no support from reading publicly published papers from other researchers, it would be tough. But with a large majority of anti-ai countries we could do a lot to prevent even covert attempts.