MIRI's Approach – Machine Intelligence Research Institute • Hacker News Board

Re: MIRI's Approach – Machine Intelligence Research Institute

#21

Post by vilhelm_s » Thu, Jul 30, 2015, 1:03 AM UTC

idlewords wrote: post #15

Are there any examples of substantive AI work to come out of MIRI? And have they succeeded at all at engaging the actual AI research community? The last time I looked at them, they were consumed with grandiose philosophical projects like "axiomatize ethics" and provably non-computable approaches like AIXI, not to mention the Harry Potter fanfic. But I'm asking this question in good faith - have things changed at MIRI…

They have produced a bunch of technical reports (linked to in a sibling comment), but so far only one of them has been published in a peer-reviewed venue (https://intelligence.org/files/ProgramEquilibrium.pdf), so I think a lot of people still doubt whether they are being productive at all. (An ordinary research lab employing that many people could clearly produce more papers; but an ordinary lab is not trying to bootstrap a new field from scratch).

On the other hand, the original post we are commenting on specifically held up AIXI as an example of the kind of thing they are trying to create, so if you don't like that, then you probably will never like MIRI's research no matter how successful they are. :)

As for engaging the AI community, the most high-profile example I know of is that Stuart Russell is apparently now concerned about the MIRI-style "value alignment problem" (https://www.quantamagazine.org/20150421-concerns-of-an-artif...) and has some DARPA grant to work on it.

Re: MIRI's Approach – Machine Intelligence Research Institute

#22

Post by [deleted] » Thu, Jul 30, 2015, 1:14 AM UTC

pjscott wrote: post #17

idlewords wrote: post #15

Are there any examples of substantive AI work to come out of MIRI? And have they succeeded at all at engaging the actual AI research community? The last time I looked at them, they were consumed with grandiose philosophical projects like "axiomatize ethics" and provably non-computable approaches like AIXI, not to mention the Harry Potter fanfic. But I'm asking this question in good faith - have things changed at MIRI…

If you'd like to give them another look, here's an up-to-date list of their publications: https://intelligence.org/all-publications/

[deleted]

Re: MIRI's Approach – Machine Intelligence Research Institute

#23

Post by guelo » Thu, Jul 30, 2015, 1:18 AM UTC

blennon wrote: post #14

reasonattlm wrote: post #10

MIRI is one of the success stories to emerge from the transhumanist community of the 80s to 00s. Others include the Methuselah Foundation, SENS Research Foundation, Future of Humanity Institute, and so on. When it comes to prognosticating on the future of strong AI MIRI falls on the other side of the futurist community from where I stand. I see the future as being one in which strong AI emerges from whole brain emula…

I agree with you in terms of approach that AI will emerge first from brain emulation. I disagree with you on the timeline. I know you say 'starting around 2030', but I think that's a little ambitious. While I'm an AI/machine learning practitioner now, my recent Ph.D. work was on computational modeling of the nervous system; namely the cerebellum. The reason I say 2030 is ambitious, is because there are still a lot of…

Seems like it would be quicker to obtain full knowledge of how DNA and cell replication work. Then the simulation could grow a brain without having to fully understand it.

Re: MIRI's Approach – Machine Intelligence Research Institute

#24

Post by dmfdmf » Thu, Jul 30, 2015, 1:53 AM UTC

I agree with their financial approach which is funded through voluntary donations, so more power to them. But then I think NASA should be funded voluntarily and would probably have a larger budget if they did. I would donate to NASA if they repudiated government money.

I reject the "how many peer-reviewed, university-associated journal articles have they published" as any measure of success. The university system is a closed guild and doomed in the internet age. I'd bet my bottom dollar the next big breakthrough in AI (or any field) comes from outside that system. Anyone who has an original, new idea to break the AI logjam (P=NP for instance) has no peers in the university system but has a handful of peers world-wide reading arxiv.org or other open forums, even HN.

As far as MIRI's AI program, I think they commit the same error as everyone else; confounding the processing of meaningless symbols (what computers can do) with actual sensory awareness of existence (what brains do). The latter is what gives the symbols meaning and humans are not threatened by the former. Few people truly understand the import of Searle's Chinese Room thought experiment and its relevance to AI and computers. But these are philosophic and epistemological questions that most people dismiss or ignore at their own peril.

Re: MIRI's Approach – Machine Intelligence Research Institute

#25

Post by ikeboy » Thu, Jul 30, 2015, 1:59 AM UTC

dmfdmf wrote: post #24

I agree with their financial approach which is funded through voluntary donations, so more power to them. But then I think NASA should be funded voluntarily and would probably have a larger budget if they did. I would donate to NASA if they repudiated government money. I reject the "how many peer-reviewed, university-associated journal articles have they published" as any measure of success. The university system is…

>humans are not threatened by the former

So you categorically reject the scenario of a machine "processing meaningless symbols" being harmful in any way? How does that follow from the Chinese Room experiment?

Re: MIRI's Approach – Machine Intelligence Research Institute

#26

Post by davmre » Thu, Jul 30, 2015, 2:00 AM UTC

dmfdmf wrote: post #24

I agree with their financial approach which is funded through voluntary donations, so more power to them. But then I think NASA should be funded voluntarily and would probably have a larger budget if they did. I would donate to NASA if they repudiated government money. I reject the "how many peer-reviewed, university-associated journal articles have they published" as any measure of success. The university system is…

> The latter is what gives the symbols meaning and humans are not threatened by the former.

For what it's worth, I think you're wildly wrong a) that the Chinese room experiment has anything profound to say about conscious experience, and b) that most AI researchers haven't thought about this.

That said, MIRI is concerned exactly with threats from machines that are very very good at processing meaningless symbols. If someone writes a simple reinforcement learning algorithm, asks it to produce paperclips, and it destroys the human race (http://wiki.lesswrong.com/wiki/Paperclip_maximizer), we're really past the point of caring whether the algorithm has awareness of its own experience. There are interesting philosophical questions there, but it's not within the domain of solving the problem MIRI cares about.

Re: MIRI's Approach – Machine Intelligence Research Institute

#27

Post by fiatmoney » Thu, Jul 30, 2015, 2:12 AM UTC

deegles wrote: post #12

reasonattlm wrote: post #10

MIRI is one of the success stories to emerge from the transhumanist community of the 80s to 00s. Others include the Methuselah Foundation, SENS Research Foundation, Future of Humanity Institute, and so on. When it comes to prognosticating on the future of strong AI MIRI falls on the other side of the futurist community from where I stand. I see the future as being one in which strong AI emerges from whole brain emula…

Your statement about needing rejuvenation treatments before we can develop strong AIs makes sense. The average age of Nobel Prize winners is increasing over time[1], we could imagine a future where this average age is greater than the average human lifespan. At that point only those scientists who have exceedingly long careers and lifespans will be able to further their respective fields. [1] http://priceonomics.com/…

"Nobel-winning scientist age" is not a good proxy for "productive scientist age", for a variety of reasons. They mention specifically lag time between discovery & recognition, but you also have issues where the "name" behind the discovery is the guy in charge of the lab, but the actual discovery (and sometimes the idea) is generated by the 30-year-old postdoc / assistant prof / etc.

There is also a sampling bias issue where scientists in academia as a whole are getting older because the boomers still have a death grip on institutional positions, and academia as a whole is shrinking.

Re: MIRI's Approach – Machine Intelligence Research Institute

#28

Post by fitzwatermellow » Thu, Jul 30, 2015, 2:23 AM UTC

We can't comprehend the danger that could be caused by strong AI because we don't understand how to make worlds. Embedded within the fabric of the cosmos are the instructions for materializing a black hole from dark matter and energy. And though the recipe may be hidden from our mortal minds it's possible they may be discovered by a de novo thinking machine. A software system with infinite IQ but the moral sense of a one day old. An electronic brain that would have no compunction in creating a super massive black hole right here on Earth just to see if it can succeed. Irrespective of the consequences to humanity.

If this "Jupiter-sized" consciousness is allowed to have thoughts that are not "amenable to inspection". Even perhaps beyond comprehension. And the danger elides to the scale of a solar system with nothing but diamonds in it. Then, by the naysayers own logic, shouldn't all AI research be outlawed? Or at least confined to the equivalent of CDC-level-5 quarantine labs?

If on the other hand you believe that Nature has certain innate prophylactics. And that it takes more than a super will to bend the laws that govern space and time. You may be self-assured in thinking we as a species have at least a century or two before we really need to begin worrying about virtual immortals.

Re: MIRI's Approach – Machine Intelligence Research Institute

#29

Post by dmfdmf » Thu, Jul 30, 2015, 2:24 AM UTC

ikeboy wrote: post #25

dmfdmf wrote: post #24

I agree with their financial approach which is funded through voluntary donations, so more power to them. But then I think NASA should be funded voluntarily and would probably have a larger budget if they did. I would donate to NASA if they repudiated government money. I reject the "how many peer-reviewed, university-associated journal articles have they published" as any measure of success. The university system is…

>humans are not threatened by the former So you categorically reject the scenario of a machine "processing meaningless symbols" being harmful in any way? How does that follow from the Chinese Room experiment?

> So you categorically reject the scenario of a machine "processing meaningless symbols" being harmful in any way?

How does this follow from what I said?

> How does that follow from the Chinese Room experiment?

The implicit (philosophic) question behind the Chinese Room experiment is; where does meaning come from? What imbues meaning to the meaningless symbols that computers process\? This is an old, unanswered philosophic question. I implied it in my answer but I will make it explicit; it comes from our perceptual awareness of existence via the senses.

Re: MIRI's Approach – Machine Intelligence Research Institute

#30

Post by dmfdmf » Thu, Jul 30, 2015, 2:43 AM UTC

davmre wrote: post #26

dmfdmf wrote: post #24

I agree with their financial approach which is funded through voluntary donations, so more power to them. But then I think NASA should be funded voluntarily and would probably have a larger budget if they did. I would donate to NASA if they repudiated government money. I reject the "how many peer-reviewed, university-associated journal articles have they published" as any measure of success. The university system is…

> The latter is what gives the symbols meaning and humans are not threatened by the former. For what it's worth, I think you're wildly wrong a) that the Chinese room experiment has anything profound to say about conscious experience, and b) that most AI researchers haven't thought about this. That said, MIRI is concerned exactly with threats from machines that are very very good at processing meaningless symbols. If…

> For what it's worth, I think you're wildly wrong a) that the Chinese room experiment has anything profound to say about conscious experience, and b) that most AI researchers haven't thought about this.

We'll just have to disagree about a) but see my answer to the other reply about the implicit question behind the CRE.

Regarding b), I never said they haven't thought about it. I said "few people truly understand" by which I meant they have failed to understand the implications and have drawn the wrong conclusions. You don't get gold stars for thinking hard.

Regarding LessWrong, all I can say is that you can't know you are "less wrong" until you know what is true. In logic you can't assume an unknowable as your standard of the true. But to reiterate my point, these are questions of philosophy and epistemology, fields which absolutely essential to the "domain of solving the problem MIRI cares about".