Live data from Hacker News

John Carmack: I’m going to work on artificial general intelligence

facebook.com

451–460 of 930 posts

Re: John Carmack: I’m going to work on artificial general intelligence

#451

Earlier quoted context omitted.

iirc in a recent talk with Joe Rogan, John mentioned something about robots doing judo...

Why do people of such intelligence subject themselves to being interviewed by dumb-as-a-rock Joe Rogan? I must admit that I often watch his interviews because he invites interesting people, but I can't help but cringe when Rogan gives his opinions.

Seriously. That guy is a stoned idiot who massively overestimates the insight of his high ramblings.

Re: John Carmack: I’m going to work on artificial general intelligence

#452

Earlier quoted context omitted.

Yeah I believe in this game / simulated world NPC idea too. To get the kind of complexity we want we either need sensors in the real world or interfacing in a virtual world that humans bring complexity to (probably both -- the humans are part of the sensing technology to start). Things like AlphaZero etc. got good cuz they had a simulatable model of the world (just a chess board + next state function in their case).…

I've long taken the position that intelligence is mostly about getting through the next 10-30 seconds of life without screwing up. Not falling down, not running into stuff, not getting hurt, not breaking things, making some progress on the current task. Common sense. Most of animal brains, and a large fraction of the human brain, is devoted to managing that. On top of that is some kind of coarse planner giving goals…

I remember this TED talk many years ago where the speaker proposes that intelligence is maximizing the future options available to you:

https://www.ted.com/talks/alex_wissner_gross_a_new_equation_...

Re: John Carmack: I’m going to work on artificial general intelligence

#453

Earlier quoted context omitted.

I'm not sure I want AGI to succeed, given some of the possibilities. Sure if it plays nicely alongside us, amplifying human society, that's great. But if we get relegated to second class with the AIs doing everything meaningful, then no thanks. But it's still a fascinating endeavor.

Why not? I'd say that a world that is managed by AGI with limited input from human beings is a good goal to have. If AGI could be done without the nasty parts of human psychology and they're inherently superior to genetically intact human beings why shouldn't be embrace it? I understand that it's a big assumption to make -- that a benevolent AI could be constructed. But under that assumption, why not have a benevolen…

Maybe. Yeah, human politics and justice systems leave something to be desired. But my worry was little bit beyond that. That the AIs would take all the meaningful work, discoveries and creativity away from us, leaving us just to amuse ourselves. Some people might be okay with that, but I don't think becoming pets is the best goal for the human race.

If the benevolent AI ruler(s) restrained themselves to allow for humans to flourish, then okay. Assuming it could be constructed benevolently.

Re: John Carmack: I’m going to work on artificial general intelligence

#454
post #311

Wow, there are a lot of people in this thread arguing whether or not John Carmack has the right skills to help AGI, or about the specifics of his knowledge. Do you all realize you're arguing about nothing? Good for him for doing something he seems excited about. Maybe we should all stop gossiping and go do something we're excited about too.

If he announced that he is retiring and sailing around the world, I’m sure you’d find people asserting that his skills as a programmer couldn’t possibly translate to being a world-class sailor. Consider for a moment that he might just be very interested in AGI and wants to work on it.

Re: John Carmack: I’m going to work on artificial general intelligence

#455

Earlier quoted context omitted.

You are saying "probably wipe out humanity" is a bad thing ? I know of several million species that would strongly disagree. Especially if AGIv1 decides to tune the genetics of say most mammals to append 'sapient' to the end of their species name. Perhaps more constructively consider that AGI is simply the next iteration of 'humanity', yea sure the old versions are redundant anarchisms and apart from some living rese…

You know of 0 species that would strongly disagree, because those several million species don't have the capacity to agree or disagree on whether "wiping out humanity is a bad thing". And you don't speak for them, no matter how much caring about the planet you think you're doing.

I absolutely speak for them, who else will ? Clearly not people like you. I have the capacity to speak for them and so I do. And I will be the only judge about how much "caring" I do or don't do.

Re: John Carmack: I’m going to work on artificial general intelligence

#456

One could imagine him beginning to work in gaming AI, but he says he wants to work on AGI, not simply gaming AI. I'm personally not convinced that it is encouraging when someone bright sets their sights on AGI, particularly someone who appears to have never competed on Kaggle. It screams hubris.

I am not convinced current AI is the approach to an AGI. I think it is at least feasible someone outside of this sphere has a reasonable shot at it. It feels like many AI researchers get caught up in refining existing techniques that amount to fancy statistics algorithms and data crunching, but not AGI. Current AI techniques may be synthesized or used in part in some AGI but it’s clear there is a revolutionary step t…

The announcement today has the potential to cut the number of years estimated (to reach AGI) in half, and I'm sure VC funds are DM'ing him non-stop right now. But we don't know which path Carmack will take, and as you rightly point out, current trends in AI is mostly ML/xNN's with a goal of turning data into profit (heavy focus on products/markets). Even those talking about AGI are fractured into different groups. Oddly, many in ML are talking about future abilities that are mostly defined by AGI research (even if they see AGI as a distraction. From his post, I don't think "product development" is a goal. Not clear what challenges or milestones he will set for himself (just having a better testing suite has become an AGI issue of late, so maybe he will contribute to that first).

Re: John Carmack: I’m going to work on artificial general intelligence

#457

This doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) As recent as his last Oculus Connect keynote, he exto…

> (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) This may be his biggest impediment. ML has gotten very far with looking at problems as linear algebraic systems, where optimizing a loss function mathematically yields a good solution to a precisely defined (and well circumscribed) classif…

Humans are not AGI's. We're specialised in human survival, not general intelligence. We're pretty limited in intelligence in many ways, actually, and the environment doesn't support generality. Without a proper challenge an agent would not become super intelligent. The cost of developing such an intelligence would conflict with the need to minimise energy for survival.

Re: John Carmack: I’m going to work on artificial general intelligence

#458

One could imagine him beginning to work in gaming AI, but he says he wants to work on AGI, not simply gaming AI. I'm personally not convinced that it is encouraging when someone bright sets their sights on AGI, particularly someone who appears to have never competed on Kaggle. It screams hubris.

I am not convinced current AI is the approach to an AGI. I think it is at least feasible someone outside of this sphere has a reasonable shot at it. It feels like many AI researchers get caught up in refining existing techniques that amount to fancy statistics algorithms and data crunching, but not AGI. Current AI techniques may be synthesized or used in part in some AGI but it’s clear there is a revolutionary step t…

> I am not convinced current AI is the approach to an AGI

This is a non-sequitor. I didn't argue that current AI theory, or Kaggle, will lead us to AGI.

> Kaggle is almost just an optimization fest

Public machine learning competitions have produced a lot of innovative learning techniques. If Kaggle is so easy, and AGI so hard, it would follow that anyone tackling AGI would have some experience applying machine learning competitively in some public space. It doesn't necessarily need to be Kaggle. Kaggle just happens to be good at hosting such public competitions, and in fact has surfaced several state-of-the-art implementations. The difference between prize money (~top 5) and no-prize-money (5-10) may be an "optimization fest", but without the competition, the solutions presented would have entered the public sphere at a much slower rate.

EDIT: Please be kind and explain your downvotes.

Re: John Carmack: I’m going to work on artificial general intelligence

#459

Earlier quoted context omitted.

I don't see why that's not GI. That HI enables us to do incredible things, far beyond what we evolved to do (we evolved to hunt and fuck and exist in small groups, not to do quantum mechanics).

It begs the question, though: is the development of AGI contingent on some magical breakthrough, or is it a matter of endless tinkering and cobbling until we've got something that works?

It's probably both. There's going to need to be a series of breakthroughs, but there's also going to be a lot of engineering required. I believe that AGI won't happen suddenly. It will require putting a lot of pieces together. We'll get systems that have an incrementally better and better model of their environment.

Personally, I find it kind of offensive how scientifically-minded people believe they have the monopoly on generating ideas and making the world progress. That's clearly not true. Deep learning research wouldn't be where it is without GPUs, and compilers like TensorFlow and PyTorch. Engineers are huge drivers of change, they make things happen.

Deep learning research already involves a lot of trial and error. Tinkering and cobbling as you put it. People can't really tell, just writing things on a whiteboard, whether it's going to work or not. There might be some mathematical intuition, but a lot of it is throwing things at the wall and seeing what sticks, empirical testing. Some very high percentage of the research being done is basically thrown away.

Furthermore, I would personally say, as someone who works in deep learning, that we're collectively getting a little myopic. We finally got neural networks to do cool things. People are very excited, but they're forgetting neural nets are not the only kind of machine learning technique around. It actually works really poorly for some things. We're just largely disregarding every other approach because we have this one cool new toy. So, I don't know, maybe the next huge breakthrough will come from someone who's a deep learning outsider, and who's not completely locked into this paradigm and unwilling to look at anything else.

Re: John Carmack: I’m going to work on artificial general intelligence

#460

This doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) As recent as his last Oculus Connect keynote, he exto…

AI today is comparable to physics in the 1700s. Back then, it was a bunch of people tinkering with prisms and apples. Today, it's a bunch of people tinkering with hyperparameters. I suspect that we know as little about AGI today as someone in the 1700s knew about QFT. Not only did they not know about QFT, but they didn't even know that they didn't know it.
Post reply on HN