Earlier quoted context omitted.
Don't be absurd. You're acting like he's Neo from The Matrix, capable of downloading kung-fu directly into his brain.
Carmack is also a good grappler.
yeap
John Carmack: I’m going to work on artificial general intelligence
681–690 of 930 posts
Re: John Carmack: I’m going to work on artificial general intelligence
#682I've been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I'll bet Carmack can offer some significant contributions. I'll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field. Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward…
For any interactive reward-giving environment E, you could place X into E and see how much reward X gets; likewise for Y. If X gets more reward, you can consider that as evidence of X being more intelligent. That's an odd definition of intelligence. By that definition, a bird is more "intelligent" than a human at the task of opening a nut. Seems like "fitness" would be a much more appropriate term. It seems especiall…
I think the idea is that you would have many environments and each one is a voter.
If an AGI candidate wins the board game vote but no others (the hunter-gathering vote, the walking and crawling vote, the "publish or perish" vote, etc), it will be trounced by something that is not quite so good at board games but is more flexible and adaptable - i.e., general.
I'm an AGI skeptic myself, but I do think that's the best attempt at a formalism for ranking AGI attempts that I've seen so far (disclaimer: as a skeptic I haven't exactly done a deep dive into the field).
Re: John Carmack: I’m going to work on artificial general intelligence
#683Earlier quoted context omitted.
You might be interested in the No Free Lunch Theorem ( https://en.wikipedia.org/wiki/No_free_lunch_theorem ). For what I skimmed from your paper, it looks like the LH agents may be viewed as discrete optimization processes trying to optimize an objective/utility function across an infinite space of possible environments (infinite voters). If it is the case, and if each environment vote has the same weight, you may be…
I think you're right to bring up the NFLT, but I don't think it is applicable, it just points at the real question. The key assumption to get the NFLT is that each environment vote has the same weight, i.e. we are targeting a uniform distribution on objective functions / environments / problems / whatever you call it. If you break this assumption, you get an opposite result which is that search algorithms divide into…
Right, the environments are not uniformly distributed. In fact, the paper actually defines not one single intelligence comparator but an infinite family, parametrized by a hyperparameter which is, essentially, a choice of which environments vote and how to count their votes. Crucially, this doesn't change the truth of the structural theorems (except that some of the theorems require the hyperparameter satisfy certain constraints).
Other authors (Legg and Hutter, 2007) followed the line of reasoning in your comment much more literally. They proposed to measure the intelligence of an agent as the infinite sum of the expected rewards the agent achieves on each computable environment, weighted by 2^-K where K is the environment's Kolmogorov complexity. Which seems as if it gives "one true measure" of intelligence, but actually that isn't the case at all, because Kolmogorov complexity depends on a reference universal Turing machine (Hutter himself eventually acknowledged how big a problem this is for his definition, Leike and Hutter, 2015).
My position is that any attempt to come up with "one true comparison of intelligence" (as opposed to a parametrized family) should be viewed with skepticism, because relative intelligence really must depend on a lot of arbitrary choices.
Re: John Carmack: I’m going to work on artificial general intelligence
#684This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. They have to operate in a world, interact with others, survive, and accomplish goals. Simulator technology is now good enough that you can do quite realistic worlds. Imagine The Sims, with a lot more internal smarts and real physics, as a base for work. Robotics has the same issue…
> This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. I don't think so. Game NPCs don't need AI, which would be way overkill; they just need to provide the illusion of agency. I think for general AI you need a field where any other option else would be suboptimal or inadequate, but in videogames general AI is the suboptimal option.…
> Game NPCs don't need AI
> ... more cost effective is to just fake it!
Many players complain in story heavy games that their choices have no consequences to the story - this is largely because building stories with meaningful branches isn't economically feasible.
A game that could make NPCs react to the what the player does dynamically while also creating a cohesive story for the player to experience would be absolutely groundbreaking in my opinion.
This is more in the realms of AI story generation but I haven't seen any work on this that generates stories you would ever mistake as coming from a human (please correct me if I'm wrong) so it would be amazing to see some progress here.
Re: John Carmack: I’m going to work on artificial general intelligence
#685Earlier quoted context omitted.
Carmack is unquestionably a genius, but I think it's quite unlikely his solo work in a new domain will leapfrog an entire field of researchers. I wouldn't, however, bet against some kind of insanely clever development coming out of his new endeavor. Something like an absurdly efficient new object classifier, that reduces the compute requirements for self-driving cars by a non-trivial factor, would be a very Carmack t…
The problem with the "field of researchers" is that most of us aren't geniuses. We're just plugging away at problems, like normal people. The opportunity for a genius is to come in, synthesize all existing information on the subject, and then come up with a novel approach to the whole thing. In some part, I think that is what Elon Musk has been able to do effectively. He comes into a field that already exists, reads…
>You can only do that effectively if you have the mental capacity to keep all that info in your head at once, I think.
Yep, plus all the different perspectives from other endeavors. Extending human memory will be a really great accomplishment with brain-computer interfaces.
Re: John Carmack: I’m going to work on artificial general intelligence
#686Earlier quoted context omitted.
I don't think that approach is going to work. For any clearly bounded and delineated task, such as a game, the most optimal, lowest energy and lowest cost solution is not AGI but a custom tuned specialist solver. This is why I don't think Deep Blue or Alphago are paths towards AGI. They are just very highly advanced single-task solvers. Now Alphago and it's implementation framework are much more sophisticated than De…
The only way i think it can be done is simulated evolution.. be that simulated evolution of neural nets or something else. As others have mentioned here though.. this becomes horrifying if we've created something sentient to kill in games or enslave.
Ted Chiang wrote an interesting novella, The Lifecycle of Software Objects, about that very subject
https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Obje...
Re: John Carmack: I’m going to work on artificial general intelligence
#687Earlier quoted context omitted.
How many of them are doing real research, though? Corporate researchers improve ads impressions and academics researches are busy generating pointless papers or they won't be paid. Very few if any do actual research.
Generating papers is research. I don't understand why you dismiss all papers as pointless.
I don’t think all papers are pointless but it’s been shown that many are not reproducible, so those are worthless and pointless. There was that guy a few months ago who tried to reproduce the results of 130 papers on financial forecasting (using ML and other such techniques) and found none of them could be reproduced and most were p-hacked or contained obvious flaws like leaking results data into the training data. An academic friend of mine who works in brain computer interfacing also says that a large number of papers he reviews are borderline or even outright fraudulent but many get published anyway because other reviewers let them through.
So I definitely wouldn’t dismiss all papers as pointless, but there certainly is a large percentage that are, enough that you can’t simply accept a published papers results without reproducing it yourself.
Re: John Carmack: I’m going to work on artificial general intelligence
#688Earlier quoted context omitted.
You might be interested in the No Free Lunch Theorem ( https://en.wikipedia.org/wiki/No_free_lunch_theorem ). For what I skimmed from your paper, it looks like the LH agents may be viewed as discrete optimization processes trying to optimize an objective/utility function across an infinite space of possible environments (infinite voters). If it is the case, and if each environment vote has the same weight, you may be…
I think you're right to bring up the NFLT, but I don't think it is applicable, it just points at the real question. The key assumption to get the NFLT is that each environment vote has the same weight, i.e. we are targeting a uniform distribution on objective functions / environments / problems / whatever you call it. If you break this assumption, you get an opposite result which is that search algorithms divide into…
Yep its clear that the NFLT only apply if we consider all possible environments equally.
In practice, we are indeed not interested in every imaginable environments, only in "realistic" ones.
It was not clear for me if the paper addressed such concerns for AGI, e.g. when writing:
To achieve good rewards across the universe of all environments, such an AI would need to have (or appear to have) creativity (for those environments intended to reward creativity), pattern-matching skills (for those environments intended to reward pattern-matching), ability to adapt and learn (for those environments which do not explicitly advertise what things they are intended to reward, or whose goals change over time), etc.
But like I said, I only skimmed it.
In general (not talking about the paper there), I have the impression that this is something that may be missed (sometimes even by researchers working in the domain), and I agree very much to your point!
This is why I think the NFLT gives us an interesting theoretical insight here:
Making a "General" AI is not actually about creating an approach that is able to learn efficiently about any type of environment.
Re: John Carmack: I’m going to work on artificial general intelligence
#689Earlier quoted context omitted.
Puzzle game with a great story. I recommend it to the HN people.
I loved it, but my issue with that game was severe motion sickness after 20-30 minutes... never finished it :(
Thanks for the warning, I cannot even play Minecraft. I wish Carmack had tackled motion sickness in VR/Games before switching to AI; he did talk about it in the interviews as being a limitation though.
There's a need gap[1] to solve Simulation Sickness in VR and First Person games.
[1]: https://needgap.com/problems/7-simulation-sickness-in-vr-and...
Re: John Carmack: I’m going to work on artificial general intelligence
#690This doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) As recent as his last Oculus Connect keynote, he exto…
> He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). You must be joking, right? I'm as much of a Carmack fan as anyone here, but overstating the skills of one personal hero does no good to anyone.
funny i wanted to make the same comment last night but was too lazy.
wasn't the first time John did what he did. and it's not the usual kind of learning either. he was learning by first principles. i truly love this idea of replaying in your own mind what went on when something was discovered (or at least come close to it).
contrast that with how ML & AI are taught nowadays: thrown into a Jupyter notebook with all FAANG libraries loaded for you...