Earlier quoted context omitted.
Counting only articles and conference papers that look like they are in at least somewhat established journals or conferences, I quickly count: 2015: 4 2014: 6 2013: 1 2012: 5 2011: 1 2010: 4 That would be about the level of 1 or 2 very mediocre early career scientists. And very little for MIRI's 8 people staff and 13 Research Associates. MIRI publishes a lot more, but they mostly operate outside of traditional scien…
How many of their staff hold PhDs, particularly in cognitive science, foundations of math, or AI? It would seem like a useful thing for them to get.
MIRI's Approach – Machine Intelligence Research Institute
71–78 of 78 posts
Re: MIRI's Approach – Machine Intelligence Research Institute
#72Earlier quoted context omitted.
How many of their staff hold PhDs, particularly in cognitive science, foundations of math, or AI? It would seem like a useful thing for them to get.
https://intelligence.org/team/ Staff: 1 out of 8 seem to have a PhD Research Associates: 3 out of 14.
Huh. They should probably increase that number.
Re: MIRI's Approach – Machine Intelligence Research Institute
#73Earlier quoted context omitted.
> You don't consider AIXI part of the actual AI research community? I just briefly tried to read up on AIXI. The concept is ...not without merit, but I think it totally sweeps the central problem (We have no idea how to program an AI) under the rug in two ways: (1) The problem of programming ...programming anything at all, is swept under the rug by assuming a formalism in which every possible algorithm/program can be…
You're right, of course; AIXI doesn't attempt to tackle the practical problem of building an AI. However, it does give us a concrete, non-hand-wavey algorithm which can reasonably be considered "AI, if we ignore resource constraints". Consider that it took ~50 years to go from the inception of AI to a formal model like AIXI; or ~40 years from the definition of NP-completeness (ie. the realisation that scalability and…
> algorithm which can reasonably be considered "AI, if we ignore resource constraints"
I am not convinced. Also the reward function is assumed to magically be given. I think half of the difficulty in any real world problem would be how to design the reward function.
Also, even when assuming we have the reward function, do we really know that choosing the action that has the best weighted reward over the set on "world model algorithms" (hypothesis), produces actions that actually are intelligent? Yes, it sounds intuitively somewhat plausible, but do we have anything better than this hunch, that it sounds kinda good? Maybe it would actually turn out to produce really silly outcomes, who knows.
I am thinking, maybe the shortest "world model algorithms", which are given the largest weight, are just mostly stupid. And there is the No Free Lunch theorem, which states that averaging over all possibilities, while may sound clever, produces just garbage (i.e. no better than random guess).
Re: MIRI's Approach – Machine Intelligence Research Institute
#74Earlier quoted context omitted.
You're right, of course; AIXI doesn't attempt to tackle the practical problem of building an AI. However, it does give us a concrete, non-hand-wavey algorithm which can reasonably be considered "AI, if we ignore resource constraints". Consider that it took ~50 years to go from the inception of AI to a formal model like AIXI; or ~40 years from the definition of NP-completeness (ie. the realisation that scalability and…
> algorithm which can reasonably be considered "AI, if we ignore resource constraints" I am not convinced. Also the reward function is assumed to magically be given. I think half of the difficulty in any real world problem would be how to design the reward function. Also, even when assuming we have the reward function, do we really know that choosing the action that has the best weighted reward over the set on "world…
> I think half of the difficulty in any real world problem would be how to design the reward function.
That's exactly what MIRI's trying to do ;)
> do we really know that choosing the action that has the best weighted reward over the set on "world model algorithms" (hypothesis), produces actions that actually are intelligent?
As far as AIXI is concerned, this is the definition of an "intelligent action": that which leads to the largest expected utility over the agent's lifetime. It's fine to disagree with this definition, but one of the reasons to define AIXI at all is to have something concrete to point at, rather than spending decades debating these sorts of quasi-philosophical questions (note that I've carefully chosen words like "can be reasonably considered" rather than "is").
> I am thinking, maybe the shortest "world model algorithms", which are given the largest weight, are just mostly stupid.
The Solomonoff prior used by AIXI dominates all computable priors; in other words, even if these world models are stupid, no computable algorithm (including humans) can do better overall.
> And there is the No Free Lunch theorem, which states that averaging over all possibilities, while may sound clever, produces just garbage (i.e. no better than random guess).
The No Free Lunch theorem is a Mathematical curiosity with no particular relevance to the world. In particular, it completely ignores computational complexity: it gives equal weight to all (computationally) simple explanations (eg. "there is a star orbiting the Earth, it will keep orbiting" and "the Earth is spinning near to a star, and will keep spinning"), as well as to all (computationally) complicated explanations (eg. "the atmosphere has been bombarded by cosmic rays which, by sheer chance, have an effect which looks like a star, but it's unlikely for that coincidence to continue" and "the Earth is spinning near to a star, but tomorrow at 13:48 GMT the Martians, who have managed to elude all of our telescopes and probes, will attack the Earth with a weapon which switches the direction of rotation"), as well as all incomputable explanations. Trying to predict anything in such situations is clearly futile, which is basically what the NFL theorem says; yet such situations can never actually arise outside of though experiments.
Although AIXI itself is incomputable, it is specifically defined to interact with computable environments, so No Free Lunch doesn't apply.
Re: MIRI's Approach – Machine Intelligence Research Institute
#75Earlier quoted context omitted.
> algorithm which can reasonably be considered "AI, if we ignore resource constraints" I am not convinced. Also the reward function is assumed to magically be given. I think half of the difficulty in any real world problem would be how to design the reward function. Also, even when assuming we have the reward function, do we really know that choosing the action that has the best weighted reward over the set on "world…
> I think half of the difficulty in any real world problem would be how to design the reward function. That's exactly what MIRI's trying to do ;) > do we really know that choosing the action that has the best weighted reward over the set on "world model algorithms" (hypothesis), produces actions that actually are intelligent? As far as AIXI is concerned, this is the definition of an "intelligent action": that which l…
> As far as AIXI is concerned, this is the definition of an "intelligent action": that which leads to the largest expected utility over the agent's lifetime.
No. Largest expected utility over the (weighted) set of all possible future timelines (i.e. hypotheses). AIXI chooses the action that gives the best average over the set of future timelines. But we only live in one timeline. Maybe an action that is very good, averaged over all possible futures, is very bad in our actual timeline?
Now we can think than an action that is good on average, maybe it is probably good in our real timeline, too. But the way AIXI gives weight to different future timelines is based on how short their MDL [1] is. Maybe this is not at all how real world works? Who knows.
A silly example: Maybe there are a lot of possible future timelines where things randomly explode. And maybe their MDL is actually shorter than for timelines where things stay stable. Then we produce a highly intelligent robot that does nothing else but seeks shelter in the nearest empty room. And this would be the "definition of intelligent action". (Defined as the maximized intelligent action over imagined future timelines where things mostly randomly explode.)
Btw, in NFL theorem, the averaging is over all possible datasets (and datasets usually describe something about past, not future), not over all possible explanations. (But yes, you can think that implicitly behind every dataset there is a multitude of world models which could have produced the dataset.)
[1] Minimum Description Length
Re: MIRI's Approach – Machine Intelligence Research Institute
#76Earlier quoted context omitted.
>However, it does give us a concrete, non-hand-wavey algorithm which can reasonably be considered "AI, if we ignore resource constraints". No, not really. "If we ignore resource constraints" is ignoring most of the problem . Using Kolmogorov complexity in the Solomonoff Measure also constitutes ignoring the problem of generalization by assuming an optimal compressor into existence, which again is an issue of the cogn…
> It's basically a grand victory for the fields of AI and Machine Learning that still tells us basically nothing about how an actually existing, embodied mind has to function Special relativity tells us basically nothing about how an actually existing, physical spaceship has to function; but it does constrain our speculation about space travel (ie. no FTL, the fact that accelerating massive objects requires more and…
> constrain
But AIXI, I don't see how it constrains anything. It introduces a classification: AIXI type algorithms, and other algorithms. But we don't really know if this is a useful segmentation of the search space, or perhaps as useless as considering the merits of red spaceships vs. spaceships painted with other color.
Re: MIRI's Approach – Machine Intelligence Research Institute
#77> One solution that the genetic algorithm found entirely avoided using the built-in capacitors (an essential piece of hardware in human-designed oscillators). Instead, it repurposed the circuit tracks on the motherboard as a radio receiver, and amplified an oscillating signal from a nearby computer.
That feeling some people get when a junebug lands on them.
Re: MIRI's Approach – Machine Intelligence Research Institute
#78Earlier quoted context omitted.
AIXI was neither invented by MIRI nor have they proved any nontrivial theorems about it. AIXI is Marcus Hutter's work: http://www.hutter1.net/ai/aixigentle.htm
Indeed! From a superficial inspection, Hutter's back catalogue features 89 * @inproceedings, 3 * @book, 12 * @techreport, 51 * @article, and 1 * @compression prize Over an elapsed period of '87 (from before he had completed his masters degree) to the present, on a published_artefacts/year basis, Hutter is more productive than MIRI.
What’s up with this publish or perish nonsense? If you are concerned that MIRI works on problems that are disconnected from reality, then read their papers and provide a thorough analysis.