Live data from Hacker News

John Carmack: I’m going to work on artificial general intelligence

facebook.com

651–660 of 930 posts

Re: John Carmack: I’m going to work on artificial general intelligence

#651
post #538
post #521

Earlier quoted context omitted.

You're assuming game AI means an agent that directly competes with the player. Plenty of games have NPCs with scripted routines, dialog, triggers, etc that could be improved either by reducing the dev cost to generate them without reducing quality or reacting to player behavior more naturally.

Except in those cases it's even more important that the NPCs don't do anything unexpected. Those NPCs are like actors in a stage play, you don't want them to come up with their own lines and confusing the audience. Don't forget there is a certain randomness with 'more natural' and with randomness you're going to invite Murphy to the party.

In many cases maybe. Personally I would love to play a game with a world inhabited by "individual" NPC AI:s, where they can influence the world as much as I can, with no specific act structure or story arc.

Some significant part of gaming is risk-free experimentation in a simulated world. The experiments possible are bounded by the simulation quality of the world. More realistic NPC behavior would open up for a lot more games.

Re: John Carmack: I’m going to work on artificial general intelligence

#652

Earlier quoted context omitted.

> It's like an image classifier that can identify an apple, but has no idea what an apple is Kind of like humans then.

A child picks up an apple but doesn't know what an apple "is". It doesn't even have the vocabulary to describe it. As adults we know what an apple is because we understand it as a concept, the ideal "apple", and can manipulate the concept into areas way outside the original concept (say, the phrase "apple of my eye").

A child comes pre-programmed to put things in their mouth. They also have very sophisticated reward functions built-in that identify tasty sugars entering their mouth.

Very quickly (assuming said child doesn't eat something too bad), in the absence of an external oracle, the child learns a very productive mental model of what an apple is.

This type of feedback loop seems eminently translatable to machine learning, assuming we can encode the concept space in a way that allows the model to be encoded and trained in a reasonable set of constraints

Re: John Carmack: I’m going to work on artificial general intelligence

#653

Earlier quoted context omitted.

I dunno. You're limiting your thought to computer science. I think it's more likely at this point in time biotech will produce an AGI, likely by accident. Worse one that competes directly with us for resource. We dont have a great mathematical description of our own intelligence, doing it for a tricked out slimemould would be just as hard.

> I think it's more likely at this point in time biotech will produce an AGI, likely by accident. Does a living thing count as AGI? In that case, I'd say that most parents are quite good at creating AGIs ;)

I think you missed the artificial part of AGI.

Re: John Carmack: I’m going to work on artificial general intelligence

#654

Earlier quoted context omitted.

> It's like an image classifier that can identify an apple, but has no idea what an apple is Kind of like humans then.

A child picks up an apple but doesn't know what an apple "is". It doesn't even have the vocabulary to describe it. As adults we know what an apple is because we understand it as a concept, the ideal "apple", and can manipulate the concept into areas way outside the original concept (say, the phrase "apple of my eye").

The child does know that the apple is a thing though. That it’s a separate object that can be carried around. Computer vision ML systems don’t even know that!

All they know is how to recognize a common pattern on a pixel grid, after seeing a large number of examples, and then draw a box around it.

The fact that a child has a body and can manipulate the world with all 5 senses working in concert should not be underestimated.

Re: John Carmack: I’m going to work on artificial general intelligence

#655

Earlier quoted context omitted.

> I am not convinced current AI is the approach to an AGI This is a non-sequitor. I didn't argue that current AI theory, or Kaggle, will lead us to AGI. > Kaggle is almost just an optimization fest Public machine learning competitions have produced a lot of innovative learning techniques. If Kaggle is so easy, and AGI so hard, it would follow that anyone tackling AGI would have some experience applying machine learni…

Ok, I'll take a shot: "it would follow that anyone tackling AGI would have some experience applying machine learning competitively in some public space". No, that would absolutely not follow. (I'm a pretty good devil's advocate, but I can't with this one.) And given AGI would come from some completely new breakthrough not related to the current practice of "machine learning", competitions may be completely moot. They…

> And given AGI would come from some completely new breakthrough not related to the current practice of "machine learning"

I’m not so sure of that. Intuitively AGI feels like being able to generalize and automatize what is already done in specialized problems, like having a meta program that that orchestrate and apply specialized subsystems, and adapt existing one. If playing Go, Starcraft, Speech recognition, Computer vision are already of the same building blocks, it feels like having a meta program that‘s just trained to recognize the type of problem and route it to the appropriate subsystem with some parameters tweaks is a path to AGI. In the dog example you don’t even need to have subsystem that are that better than humans individually.

Edit: my point is I feel like AGI is the interface and orchestration between specialized subsystems we already know how to create. Trying to train a big network like generalizing Alpha Go is a dead-end, but having simpler sub networks ready to be trained at a specific problem seems feasible. Much like the brain is at first seen like a big network, but in practice there are specialized areas. The key is how are these networks interfaced and which information they exchange to self adapt. Maybe these interfaces themselves are sub networks specialized in the problem of interfacing and “tuning hyperparameters”.

In short: I think when we’ll figure out how to automate Kaggle competitions (recognize the pattern of the problem, then instantiate and train the relevant subsystem) we’ll be a good step forward AGI. We don’t need better performance e.g. in image recognition, just how to figure orchestration.

Re: John Carmack: I’m going to work on artificial general intelligence

#656

Earlier quoted context omitted.

He made it cool to drive an electric car.

He shifted a whole industry towards a new paradigm. Look at Germany, they are desperate to catch up with Tesla, finally moving into electric cars. Without Elon they would keep selling their Diesel scam for the next decades.

Actually it was a bunch of Phd students from some Californian Uni that discovered the vw diesel scam. There is a short documentary about them online. Elon has no credit whatsoever in dieselgate.

However, he did make electric cars something an average person would like to have. He also chose to make it work using the same inefficient principle of hauling 2 tons of steel to transport a single person. What he made is an electric luxury car, not a car for the masses that can replace average Joe's car. Is there anything wrong with that? No, there isn't, but let's not pretend a $35k (in US - much more in EU) car that requires hours of charging after driving 250 miles unless you happen to have Tesla's superchargers on your way is a new "volkswagen - a people's car". Also I find it disingenuous to advertise full battery capacity while at the same time recommending people use only 60% of it "for longevity".

Many people don't buy new cars, but choose to buy 5-8 year old cars that are really good value if they were maintained well. It remains to be seen how Teslas behave in that market.

It would be really revolutionary if someone could create and market an electric car that was truly innovative for example: much lighter than current cars while still being safe during collision, use fuel cell technology with fuel such as methanol or similar that can be created in a sustainable way, even using a fuel cell with mined hydrocarbons and electric drive would provide for a huge reduction in emissions due to increase in efficiency.

Do Teslas have a role to play in reducing emissions? Yes, definitely, but let's not present them as a single solution to all individual transport problems.

Re: John Carmack: I’m going to work on artificial general intelligence

#657
post #650
post #596

Earlier quoted context omitted.

I don't think that approach is going to work. For any clearly bounded and delineated task, such as a game, the most optimal, lowest energy and lowest cost solution is not AGI but a custom tuned specialist solver. This is why I don't think Deep Blue or Alphago are paths towards AGI. They are just very highly advanced single-task solvers. Now Alphago and it's implementation framework are much more sophisticated than De…

> AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them. That's not the same thing. It's not approaching chess or Go as an intelligent thinking being, learning the rules and working out their consequences There are indeed some people who learn chess by "reading the manual". Or learn a language by memorizing grammar rules. Or learn h…

Alphago and it's kind are doing some things that we do, for sure. We do utilise pattern recognition and some of the neurological tools we bring to bear on these problems might look a bit like Alphago.

The thing is those are parts of our neurology that have little to do with general intelligence. I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level general intelligence machinery. In that sense high level Go and Chess players turn themselves into single-task solvers. They're better at bringing that experience and capability to bear in other domains, because they have general intelligence with which to do so, but those specialised capabilities aren't what make them a being with general intelligence. Or if specialising systems are important to general intelligence, it's as just a part of a much broader and more sophisticated set of neurological systems.

Re: John Carmack: I’m going to work on artificial general intelligence

#658

Earlier quoted context omitted.

I can trust John Carmack's words when he says in an interview, or on stage. There's a passion in his talks, a nervousness in blurting what he really feels, and those are really good traits, in my mind. I genuinely felt a sense of disappointment when he moved to Facebook (via the Occulus acquisition). So yea, fuck you, Facebook and your manipulative, life values corrupting and PR machinery. I place John Carmack miles…

I have to admit, I felt a bit disappointed too. Carmack and Facebook always struck me as an antithetical pairing - the creativity/independence of the former didn't seem to sit right with the maniacal/emotional exploitation of the latter.

I think Carmack just doesn't give a flying f* about Facebook or etc. he is interested in tech and he clearly works on stuff he is passionate about. He worked on VR and not work for Facebook. Facebook just happened to be paying for it.

Re: John Carmack: I’m going to work on artificial general intelligence

#659
post #187

Earlier quoted context omitted.

Current ML technology probably has little or nothing to do with whatever technology will eventually be needed to produce true AGI.

As I like to say: lots of people are working on making a car that is smart enough to drive itself wherever a human wants to go. How many people are working on a car smart enough to tell humans to fuck off, it doesn’t feel like driving anywhere today?

When I was working at TomTom, they didn't appreciate my proposal to develop the TomTomagotchi:

A Personal Navigation Device with a simulated personality that begs you to drive it all around town to various points of interest it desires to visit in order to satisfy its cravings and improve its mood.

I'm sure there's a revenue model getting drive through Burger Kings and car washes to pay for product placements.

Re: John Carmack: I’m going to work on artificial general intelligence

#660
post #619
post #319

I've been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I'll bet Carmack can offer some significant contributions. I'll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field. Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward…

You might be interested in the No Free Lunch Theorem ( https://en.wikipedia.org/wiki/No_free_lunch_theorem ). For what I skimmed from your paper, it looks like the LH agents may be viewed as discrete optimization processes trying to optimize an objective/utility function across an infinite space of possible environments (infinite voters). If it is the case, and if each environment vote has the same weight, you may be…

I think you're right to bring up the NFLT, but I don't think it is applicable, it just points at the real question.

The key assumption to get the NFLT is that each environment vote has the same weight, i.e. we are targeting a uniform distribution on objective functions / environments / problems / whatever you call it.

If you break this assumption, you get an opposite result which is that search algorithms divide into some equivalence classes determined by the sets of different outcomes (traces, if I remember the theorem's description) that you discriminate between.

A uniform distribution like this is actually a very very strong precondition; it implies (looking at results about the complexity of sets of strings, since choosing an environment is like choosing a string from 2^N given some encoding, etc) that you care equally about a very large number of environments most of which have no compressible structure or equivalently have a huge kolmogorov complexity. Most of these environments would not have a compact encoding, relative to a particular choice of machine, but we are weighing these the same as those environments which are actually implementable using less than a ridiculous amount of storage to represent the function.

The reason why I think this is too strong an assumption to use is then that we don't care about all these quadrillion problems which have no compact encoding - we know this because we literally can't encounter them as they would be too large to ever write down using ordinary matter.

Allowing for this, talking usefully about evaluating an AGI or equivalently a search strategy or optimization algorithm implies having an understanding of the distribution of environments / problems we care about. I think capturing this concept in a 'neat' way would be a significant contribution; I had a go during my PhD but failed to get anywhere. Unfortunately things like K-complexity are uncomputable, so reasoning about distributions in those terms is a dead-end.

Post reply on HN