John Carmack: I’m going to work on artificial general intelligence • Hacker News Board

Re: John Carmack: I’m going to work on artificial general intelligence

#741

Post by mrfusion » Thu, Nov 14, 2019, 2:20 PM UTC

whack wrote: post #650

simonh wrote: post #596

Earlier quoted context omitted.

I don't think that approach is going to work. For any clearly bounded and delineated task, such as a game, the most optimal, lowest energy and lowest cost solution is not AGI but a custom tuned specialist solver. This is why I don't think Deep Blue or Alphago are paths towards AGI. They are just very highly advanced single-task solvers. Now Alphago and it's implementation framework are much more sophisticated than De…

> AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them. That's not the same thing. It's not approaching chess or Go as an intelligent thinking being, learning the rules and working out their consequences There are indeed some people who learn chess by "reading the manual". Or learn a language by memorizing grammar rules. Or learn h…

Really most people learn by a mix of the two. It’s going to take you a lot longer to learn chess is you have no clue on the rules.

But on the other hand if you get into rote memorization before you start the game it’s going to slow you down by having no context.

Re: John Carmack: I’m going to work on artificial general intelligence

#742

Post by the_af » Thu, Nov 14, 2019, 2:21 PM UTC

Aeolun wrote: post #669

the_af wrote: post #665

Earlier quoted context omitted.

> This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. I don't think so. Game NPCs don't need AI, which would be way overkill; they just need to provide the illusion of agency. I think for general AI you need a field where any other option else would be suboptimal or inadequate, but in videogames general AI is the suboptimal option.…

If you have general AI of enough sophistication to use in a game, haven’t you just created a virtual human?

Yes, probably. At which point, the "videogame" part is irrelevant.

Re: John Carmack: I’m going to work on artificial general intelligence

#743

Post by negamax » Thu, Nov 14, 2019, 2:24 PM UTC

I have seen what happens next in Blackmirror!

Re: John Carmack: I’m going to work on artificial general intelligence

#744

Post by Quarrelsome » Thu, Nov 14, 2019, 2:27 PM UTC

pingyong wrote: post #623

simonh wrote: post #596

Earlier quoted context omitted.

I don't think that approach is going to work. For any clearly bounded and delineated task, such as a game, the most optimal, lowest energy and lowest cost solution is not AGI but a custom tuned specialist solver. This is why I don't think Deep Blue or Alphago are paths towards AGI. They are just very highly advanced single-task solvers. Now Alphago and it's implementation framework are much more sophisticated than De…

I think if you have a framework that can produce arbitrary single-task solvers (which AlphaZero can't yet), you would have something indistinguishable from AGI, since communication between single-task solvers is also kinda just a single-task solver. It's certainly not the most efficient way to use our current hardware, and it's not clear to me how big some of these neural nets would have to be, but if we had computer…

> since communication between single-task solvers is also kinda just a single-task solver.

It would be nice if it worked like that but I think you're massively underestimating the problem set here. I'd suggest its more like the architectural glue one needs as an engineer writing a command line util and a fully fledged piece of Enterprise solution (i.e. orders of magnitude).

Of course because we don't actually know how intelligence exactly works we're both guessing here.

Re: John Carmack: I’m going to work on artificial general intelligence

#745

Post by Quarrelsome » Thu, Nov 14, 2019, 2:32 PM UTC

olalonde wrote: post #734

arethuza wrote: post #611

Earlier quoted context omitted.

Reverse engineering a human mind would be another approach.

This is often overlooked but it's the only approach that is pretty much guaranteed to succeed given enough time. That said, it's also likely that AGI will come about way earlier from another approach (just as planes came before the "robot birds").

While I agree this leads to haunting outcomes. e.g. If we create a successful interface then whats the point of building our own digital pastiches if we can just strap in the real thing?

Re: John Carmack: I’m going to work on artificial general intelligence

#746

Post by goldenkey » Thu, Nov 14, 2019, 2:33 PM UTC

TeamSlytherin wrote: post #463

sullyj3 wrote: post #309

Earlier quoted context omitted.

you're possibly thinking of the problem of consciousness, which is a totally separate thing. AGI is just what it says on the tin - a general intelligence. That is, a problem solver that can operate at a human or greater level in a broad variety of domains. This ability is plausibly totally orthogonal to "having the lights on" - having subjective experience.

That's just what an ML expert would say ;p Problem solvers, maximizers, and utility functions all go in the waste bin when working on AGI. And the problem peals away into other large "hard problems," like the nature of consciousness. NLP can just follow rules, but language understanding (before even reaching some general, high school level), requires knowledge outside of language itself. That leads to questions about…

The kind of AI being trained now aren't given the mechanisms of data space traversal/attention. Recently, attention mechanisms are being focused on by google. An AGI needs to learn that it can affect the system - dependent decision theory factors in here too.

Also, growth may be hugely important. Babies start out with fuzzy learning, almost as if the learning rate starts out very small which normalizes the lack of knowledge and elevated novelty/variance of the environment.

AGI is all about predicting future utility given a circular dependency between the agent and environment. QM says we can't solve this exactly.. it's a two object interaction.. no way to gain the joint state, the ground truth, assumptions always have to be made to approximate independence.

Re: John Carmack: I’m going to work on artificial general intelligence

#747

Post by playing_colours » Thu, Nov 14, 2019, 2:35 PM UTC

Based on the interviews with Jon Carmack, he is very intelligent, realistic, reflective. Therefore, he realistically estimates his skills and the complexity of a problem. He said he would possible make an impact - and he can - by potentially attracting more attention to the field from researchers and investors, by building tools, that would help in researching the problem, by, at least, showing promising paths. He may start to dig into the problem in “Victorian Gentleman Scientist” style, but then attract investments and scale up. We can speculate and guess - time will show, fingers crossed.

This “Victorian Gentleman Scientist” style is something I am longing for. I cannot go back to the academia now with family, or spend large chunks of my time on any research, but I really want to be able to afford it. Sure, most probably, I ll become soon disillusioned with the routine of a researcher, or jump between topics of research, or just did not contribute anything meaningful, but I'd really wish there was a possibility for me, other people to afford such lifestyle.

Re: John Carmack: I’m going to work on artificial general intelligence

#748

Post by randomsearch » Thu, Nov 14, 2019, 2:35 PM UTC

Strom wrote: post #731

randomsearch wrote: post #703

Earlier quoted context omitted.

No, I mean scientific. What scientific work has Carmack done? I'm genuinely interested, because someone called him a scientist but I thought he was an engineer.

Are you in doubt that Carmack has used the scientific method to do anything? [1] If not, does your definition of scientist require something other than doing work using the scientific method? Perhaps some specific quantity of work? -- [1] One of his companies, Armadillo Aerospace, was pretty much just a series of scientifc experiments. https://en.wikipedia.org/wiki/Armadillo_Aerospace

Thanks for the explanation. We could debate whether that’s science or not, but I don’t think it’d be particularly productive and we’ve already gone a bit off topic.

Final thought from me - I was thinking about your post and it is indeed difficult to discern science from engineering. One dichotomy that occurred to me (which may not hold under close scrutiny) is that scientists are interested in _the pursuit of truth_, whereas engineers are interested in _building things_.

Re: John Carmack: I’m going to work on artificial general intelligence

#749

Post by simonh » Thu, Nov 14, 2019, 2:41 PM UTC

dangerface wrote: post #738

simonh wrote: post #657

Earlier quoted context omitted.

Alphago and it's kind are doing some things that we do, for sure. We do utilise pattern recognition and some of the neurological tools we bring to bear on these problems might look a bit like Alphago. The thing is those are parts of our neurology that have little to do with general intelligence. I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level genera…

> AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them. When given a problem it has never seen before it was able to acquire knowledge of the problem and then apply that knowledge as to solve the problem. Thats the definition of learning and intelligence that can generally be applied to any problem. > I think it's becoming clear th…

>When given a problem it has never seen before it was able to acquire knowledge of the problem and then apply that knowledge as to solve the problem. Thats the definition of learning and intelligence that can generally be applied to any problem.

It can't be applied to any problem though. Take the example i gave elsewhere of a game where you provide the rules, and as the game progresses the rules change. There are real games that work like this, generally card games where the cards contain the rules, so as more cards come into play the rules change. Alpha Zero cannot play such games, because there isn't even a way to provide it with the rules.

>> I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level general intelligence machinery. > >What are you on about? Cognition and intelligence are the same thing if it's capable of cognition or as you put it applying "cognitive tools" then it's capable of intelligence.

I'm saying that human minds apply many cognitive tools, and that Alphago is like one of those tools. It's not like the part choosing and deploying those tools, which is the really interesting and smart part of the system.

The human brain consists of a whole plethora of different cognitive mechanisms. Cognition is a broad term of a huge variety of mechanisms, none of which by themselves constitute all of intelligence. A lot of people look at Alphago and say aha, that's intelligence because it does something we do. Yes, but it only does a tiny, specialist fragment of what we do, and not even one of the most interesting parts.

Re: John Carmack: I’m going to work on artificial general intelligence

#750

Post by seanwilson » Thu, Nov 14, 2019, 2:41 PM UTC

the_af wrote: post #740

seanwilson wrote: post #684

Earlier quoted context omitted.

> Game NPCs don't need AI > ... more cost effective is to just fake it! Many players complain in story heavy games that their choices have no consequences to the story - this is largely because building stories with meaningful branches isn't economically feasible. A game that could make NPCs react to the what the player does dynamically while also creating a cohesive story for the player to experience would be absolu…

Agreed about the meaningful choices and dynamically generated reactions from NPC, but general AI is not needed for this in my opinion. You also have to consider whether the complaints of "many" players matter when publishing a game. A percentage of vocal players will complain no matter what . Yes, they will complain even if you somehow implement true AI!

> Agreed about the meaningful choices and dynamically generated reactions from NPC, but general AI is not needed for this in my opinion.

Maybe, but it would be an impressive demonstration of AI that would be very different to what has shown for Go, Chess and StarCraft.

I think a compelling AI written short story for example would be leagues ahead of what is required to write a convincing chatbot e.g. you need an overarching plot, subplots, multiple characters interacting in the world, having to track characters beliefs and knowledge, tracking what the reader must be thinking/feeling.

It would likely rely a lot on understanding real-world and culture knowledge though - Go and StarCraft are much cleaner in comparison.

> A percentage of vocal players will complain no matter what.

Yep, but I can't think of a single game that has a plot that meaningfully adapts to how the player plays. Either there's many endings but the path to get to each is short, or all the choices converge quickly back into the same path.

Again, please correct me if I'm wrong but I've look quite hard for examples of innovation in the above recently and haven't found much. You can find examples of papers on e.g. automated story generation or game quest generation on Google Scholar from the last 10 years but the examples I found weren't that compelling.