John Carmack: I’m going to work on artificial general intelligence • Hacker News Board

Re: John Carmack: I’m going to work on artificial general intelligence

#751

Post by losvedir » Thu, Nov 14, 2019, 2:45 PM UTC

Animats wrote: post #437

nikki93 wrote: post #422

Earlier quoted context omitted.

Yeah I believe in this game / simulated world NPC idea too. To get the kind of complexity we want we either need sensors in the real world or interfacing in a virtual world that humans bring complexity to (probably both -- the humans are part of the sensing technology to start). Things like AlphaZero etc. got good cuz they had a simulatable model of the world (just a chess board + next state function in their case).…

I've long taken the position that intelligence is mostly about getting through the next 10-30 seconds of life without screwing up. Not falling down, not running into stuff, not getting hurt, not breaking things, making some progress on the current task. Common sense. Most of animal brains, and a large fraction of the human brain, is devoted to managing that. On top of that is some kind of coarse planner giving goals…

That's kind of the thesis Rodolfo Llinas puts forward in a book of his, I of the Vortex[0], although more about consciousness than intelligence. That is, consciousness is the machinery that developed in order for us to predict the next short while and control our body through it.

[0] https://mitpress.mit.edu/books/i-vortex

Re: John Carmack: I’m going to work on artificial general intelligence

#752

Post by CivBase » Thu, Nov 14, 2019, 2:45 PM UTC

Ultimatt wrote: post #511

CivBase wrote: post #366

Earlier quoted context omitted.

Wireless teathering is the future of VR. True mobility is pointless when you're realistically restricted to a dedicated space anyways.

You should take more long haul flights if you think mobile is pointless.

I said mobility is pointless, not mobile. That's an important distinction.

By mobility, I mean the ability to throw the headset around and walk anywhere without worrying about leaving the range of your tether. That sort of thing is important for AR, but I just don't see it mattering for VR in the long run.

There is definitely a market for VR headsets for content delivered by a phone or builtin hardware. Those devices will realistically be limited to seated or standing-room-only experiences, though.

Re: John Carmack: I’m going to work on artificial general intelligence

#753

Post by patentatt » Thu, Nov 14, 2019, 2:46 PM UTC

shurcooL wrote: post #728

hans1729 wrote: post #2

Heh, from the comments: > Congratulations on the new project, and may your hubris not doom us all.

I wanted to find some discussion on this topic. I’m not pessimistic, but I am curious to consider what it may mean for humanity if/when AGI happens.

My pet theory is that anything we regard as truly intelligent has to have a sense of self, an identity if you will. In addition, it has to be motivated by something. All living things, and all the intelligent living things, have a survival and reproduction instinct on some level which is an animating force driving higher level thinking and actions. Basically, any AGI has to have something approaching a ‘soul’ to be ‘intelligent.’ And if that emerges, combined with exponential iterative evolution that isn’t limited by biology ... well all the sci-fi tropes seem plausible.

Re: John Carmack: I’m going to work on artificial general intelligence

#754

Post by goldenkey » Thu, Nov 14, 2019, 2:49 PM UTC

simonh wrote: post #708

goldenkey wrote: post #700

Earlier quoted context omitted.

Ah, I misread. So we agree then. On the other hand, I would not be surprised if the hippocampus was highly developed in chess players like Bobby Fischer which could translate into better spatial reasoning. Perhaps general intelligence is best trained by variance.... Not targeted training.

You could be right, and for the record I upvoted your comment for the contribution regarding you experience with high level chess players. I think the downvotes you’re getting are regrettable.

I appreciate your sentiment. This field is my focus right now. My bachelors is in BioChemMed but I am doing a master in CS and have finished many courses including the free ones by Hinton, LeCun, and Bengio.

Here is my strongest prediction:

AGI is only possible if the AGI is allowed to cause changes to its inputs.

Current ML needs to be grafted towards attention mechanisms and more boltzmann net / finite/infinite impulse response nets.

Re: John Carmack: I’m going to work on artificial general intelligence

#755

Post by ufmace » Thu, Nov 14, 2019, 2:52 PM UTC

fastball wrote: post #363

mortenjorck wrote: post #178

Earlier quoted context omitted.

Carmack is unquestionably a genius, but I think it's quite unlikely his solo work in a new domain will leapfrog an entire field of researchers. I wouldn't, however, bet against some kind of insanely clever development coming out of his new endeavor. Something like an absurdly efficient new object classifier, that reduces the compute requirements for self-driving cars by a non-trivial factor, would be a very Carmack t…

The problem with the "field of researchers" is that most of us aren't geniuses. We're just plugging away at problems, like normal people. The opportunity for a genius is to come in, synthesize all existing information on the subject, and then come up with a novel approach to the whole thing. In some part, I think that is what Elon Musk has been able to do effectively. He comes into a field that already exists, reads…

I don't see Elon as a genius at any kind of engineering. Everything he's done there was pretty easily foreseeable as being physically possible. What he is remarkably good at is selecting daring and potentially market-changing business goals, and executing against them consistently and aggressively despite naysayers.

It's easy to say that it's probably possible to land a orbital rocket first stage. But who would bet a multi-billion dollar business on being able to not only do it, but save money by doing it, when nobody had ever done it before?

Similarly, electric cars were far from new. Nobody seemed much inclined to build one that was actually a luxury car, instead of a toy for engineer-types who could put up with driving weird things. Any of the big manufacturers could have done it, and easily absorbed the losses if it failed, but none did. Elon made a wild bet on that, making a company that made nothing else, so the whole thing would go down the tubes if the idea flopped. Instead it seems to have worked. Although it seems to be harder than he anticipated, and maybe outside his skillset, to run an organization that does real mass-production.

Re: John Carmack: I’m going to work on artificial general intelligence

#756

Post by larkery » Thu, Nov 14, 2019, 2:57 PM UTC

xamuel wrote: post #683

larkery wrote: post #660

Earlier quoted context omitted.

I think you're right to bring up the NFLT, but I don't think it is applicable, it just points at the real question. The key assumption to get the NFLT is that each environment vote has the same weight, i.e. we are targeting a uniform distribution on objective functions / environments / problems / whatever you call it. If you break this assumption, you get an opposite result which is that search algorithms divide into…

Right, the environments are not uniformly distributed. In fact, the paper actually defines not one single intelligence comparator but an infinite family, parametrized by a hyperparameter which is, essentially, a choice of which environments vote and how to count their votes. Crucially, this doesn't change the truth of the structural theorems (except that some of the theorems require the hyperparameter satisfy certain…

Hah, interesting - this is a reference I hadn't seen and I like the sound of it. There was me thinking I'd had an idea of my own one time!

The reference machine thing would be the next problem to argue if using 2^-K as the weight; whilst you can make the K-complexity of any particular string low by putting an instruction in your machine that is 'output the string', this is clearly cheating! So there ought to be a connection between the reference machine and some real physics, since we are perhaps not interested in building optimisers that perform well in universes whose physics is very different to ours.

Sadly even if this were cracked I think the fact that K is uncomputable would make the result likely to be useless in practise.

Thanks for your interesting reply, I enjoyed it.

Re: John Carmack: I’m going to work on artificial general intelligence

#757

Post by jjoonathan » Thu, Nov 14, 2019, 2:58 PM UTC

foobiekr wrote: post #721

I’ve worked with legends in a specific space that is less consumer-y than games and graphics and less weirdly-desperate attention-seeking than ML and so the people therein are every bit Carmack-level but less visible. As they have aged out and wealthy’d out of working time, almost all of them have chosen to retire. These are extremely high performing individuals who have made global impact. Shutting down for people l…

I don't see any reason to be so dismissive.

Scientific pursuits have an extremely steep risk profile and are systematically underfunded because nobody knows how to capture the value that comes out of them. If someone wants to chase one on their own dime, we should celebrate that contribution to society rather than dwell on the fact that the median (even 99th percentile) outcome is that the project goes nowhere and the person continues on to retirement proper. The mean outcome could be very different and the benefit of the doubt costs us nothing, so why not give it?

Re: John Carmack: I’m going to work on artificial general intelligence

#758

Post by JohnJamesRambo » Thu, Nov 14, 2019, 2:58 PM UTC

whymauri wrote: post #58

JohnJamesRambo wrote: post #52

Earlier quoted context omitted.

You guys really get high on your own supply don't you? It's a toss-up these days whether something from computer science or a programmer will either enhance or erode the human experience and quality of life.

I'm generally inclined to agree with your first statement, but not very sure what's the point of your second statement. There was a 50/50 chance 100 years ago that discoveries in chemistry or physics were immediately weaponized. I'd say GP's analogy still holds, yeah?

Even a decade ago, I'd be all onboard seeing computer science as "advancing humanity." Lately not so much so. Maybe the comparison to phsyics is appropriate. We have reached the "nuclear weapons" age of computer science where we need programmers to make ethical stands, lest we severely damage the human condition.

Re: John Carmack: I’m going to work on artificial general intelligence

#759

Post by larkery » Thu, Nov 14, 2019, 3:00 PM UTC

jaster wrote: post #688

larkery wrote: post #660

Earlier quoted context omitted.

I think you're right to bring up the NFLT, but I don't think it is applicable, it just points at the real question. The key assumption to get the NFLT is that each environment vote has the same weight, i.e. we are targeting a uniform distribution on objective functions / environments / problems / whatever you call it. If you break this assumption, you get an opposite result which is that search algorithms divide into…

Yep its clear that the NFLT only apply if we consider all possible environments equally. In practice, we are indeed not interested in every imaginable environments, only in "realistic" ones. It was not clear for me if the paper addressed such concerns for AGI, e.g. when writing: To achieve good rewards across the universe of all environments, such an AI would need to have (or appear to have) creativity (for those env…

Yes - I think you're right that the actual interesting result from NFLT is not that 'optimisation is impossible', but that 'uniform priors are stupid'.

Re: John Carmack: I’m going to work on artificial general intelligence

#760

Post by mrd999 » Thu, Nov 14, 2019, 3:01 PM UTC

bildung wrote: post #633

sildur wrote: post #621

Earlier quoted context omitted.

Yes, but you did not manipulate the concept. You did not invent that phrase, but simply learned its meaning. A machine can do that.

Every child reinvents the concept. It wasn't there on birth, and the word and phrases didn't contain it. That's a bit difficult topic to wrap ones head around, but it is critically important to differ between signified (the concept) and signifier (the words etc.). The child develops concepts and is able to create and evaluate inferences , and thus able to understand metaphors etc. The concept is what most AI approach…

You're the only person who mentioned metaphors here. My intuition tells me metaphors will be key to developing AGI. Metaphors literally generalize; they predict; they organize and catalogue. Formation, testing, and introspection of metaphors seems to be a way forward.