John Carmack: I’m going to work on artificial general intelligence • Hacker News Board

Re: John Carmack: I’m going to work on artificial general intelligence

#891

Post by nwallin » Fri, Nov 15, 2019, 6:23 AM UTC

seanwilson wrote: post #684

the_af wrote: post #665

Earlier quoted context omitted.

> This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. I don't think so. Game NPCs don't need AI, which would be way overkill; they just need to provide the illusion of agency. I think for general AI you need a field where any other option else would be suboptimal or inadequate, but in videogames general AI is the suboptimal option.…

> Game NPCs don't need AI > ... more cost effective is to just fake it! Many players complain in story heavy games that their choices have no consequences to the story - this is largely because building stories with meaningful branches isn't economically feasible. A game that could make NPCs react to the what the player does dynamically while also creating a cohesive story for the player to experience would be absolu…

You're talking about different problems.

Story AI is basically having a writer sit down and writing a branching story tree with writing the whole way. At best it's a manually coded directed acyclic graph.

Tactical AI, ie having the bad guy soldiers move about the battlefield and shooting back at you in a realistic manner is 100% about faking it. It's better to despawn non-visible and badly placed enemies and spawn well placed non-visible enemies than have some super smart AI relocate the badly placed enemies into better locations. It's better to have simple mechanisms that lead to difficult to understand behavior than complex behavior that leads to instinctive behavior.

There was an amazing presentation at gdc maybe 3 years ago that perfectly articulated this. The game was something about rockets chasing each other. I wish I could find the link.

Re: John Carmack: I’m going to work on artificial general intelligence

#892

Post by rfhjt » Fri, Nov 15, 2019, 6:29 AM UTC

p1esk wrote: post #884

rfhjt wrote: post #883

Earlier quoted context omitted.

This is what I meant. In our society, only very few, usually already rich, can try their own ideas. Most of us have to stick with known ideas that bring profit to business owners or meaningful visibility to universities. When I was in college, I had to work on ideas approved by my professor. Now I have to work on ideas approved by my corporation. But if I had money, I'd work on something completely different. Sure, i…

What would you work on if you could? Would you say you deserve to be paid for 5 years of uninterrupted research? Do you think you have a decent chance to make a breakthrough in some field? These are the questions I ask myself.

I have some interesting ideas about managing software complexity in general (i.e. why this complexity inevitably snowballs and how we could deal with that), or about a better way to surf the internet (which may be a really big idea, tbh). But all these are moonshot ideas that gave a slim chance of success, while I need to pay ever raising bills. On the other hand I have a couple solid money making business ideas that I'm working on and that will bring me a few tens of millions, bit will be of no use to society, and I have a fallback plan: a corporate job with outstanding pay, but that brings exactly nothing to this world (it's about reshaping certain markets to make my employer slightly richer).

Do I deserve to be paid for 5 years for something that may not work? "Deserving" something doesn't have much meaning: we, the humans, merely transform solar energy into some fluff like stadiums and cruise ships. Getting paid just means getting a portion of that stream of solar energy. There is no reason I need to "deserve it" as it's unlimited and doesn't belong to anyone. A better question to ask is how can we change our society so that all, especially young, people would get a sufficient portion of resources to not think about paying bills.

Chances to make a breakthru are small, but that doesn't matter. It's a big numbers game: if chances are 1 to million, we let 1 billion people try and see 1000 successes. The problem currently is that we have these billions of people, but they are forces by silly constraints of our society to non stop solve fictional problems like paying rent.

Re: John Carmack: I’m going to work on artificial general intelligence

#893

Post by Iv » Fri, Nov 15, 2019, 6:38 AM UTC

anonytrary wrote: post #460

carlosdp wrote: post #68

This doesn't surprise me at all. He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago to dive in to AI (that's all this guy needs to become pretty damn proficient). (edit: I'm not claiming he's a field expert in a week guys, just that he can probably learn the basics pretty fast, especially given ML tech shares many base maths with graphics) As recent as his last Oculus Connect keynote, he exto…

AI today is comparable to physics in the 1700s. Back then, it was a bunch of people tinkering with prisms and apples. Today, it's a bunch of people tinkering with hyperparameters. I suspect that we know as little about AGI today as someone in the 1700s knew about QFT. Not only did they not know about QFT, but they didn't even know that they didn't know it.

Too many people make this mistake of conflating machine learning with AI. I hope someone as external to the field as Carmack will see the value of rule based inference as well. The Good Old Fashioned AI as it used to be called

Re: John Carmack: I’m going to work on artificial general intelligence

#894

Post by simonh » Fri, Nov 15, 2019, 7:10 AM UTC

marcus_holmes wrote: post #715

simonh wrote: post #667

Earlier quoted context omitted.

How would a single-task solver, or hierarchy of them, go about constructing a conceptual model of a new problem domain? The problem with a solver is it only really goes in a single direction, but when modeling a system you spend a huge amount of time backtracking and eliminating elements that yielded progress at first but then proved to be obstacles to progress later. You also need to be able to rapidly adapt to chan…

they'd do it like we do: by comparing the new situation to previous ones we know about, and applying the model that fits best, and then adapting to results. For humans, the more previous ones we know about, the better, because we have more chance of applying a model that works in the new environment. That's called "experience".

That’s a very broad, general description of behaviour that doesn’t actually describe an implementation. In fact it could apply to many completely different possible implementations. I suspect though that humans do more than this, that we have a way of either constructing entirely new models from scratch, or of dramatically adapting models to new situations without mere iterative fitting to feedback. Humans are actually capable of reasoning effectively about entirely new ideas, scenarios and problems. We have little to no idea how we do this.

Re: John Carmack: I’m going to work on artificial general intelligence

#895

Post by bildung » Fri, Nov 15, 2019, 8:00 AM UTC

mrd999 wrote: post #760

bildung wrote: post #633

Earlier quoted context omitted.

Every child reinvents the concept. It wasn't there on birth, and the word and phrases didn't contain it. That's a bit difficult topic to wrap ones head around, but it is critically important to differ between signified (the concept) and signifier (the words etc.). The child develops concepts and is able to create and evaluate inferences , and thus able to understand metaphors etc. The concept is what most AI approach…

You're the only person who mentioned metaphors here. My intuition tells me metaphors will be key to developing AGI. Metaphors literally generalize; they predict; they organize and catalogue. Formation, testing, and introspection of metaphors seems to be a way forward.

If you are interested in this direction of research: There is a big body of work regarding human cognitive processes and the role of metaphor. I would suggest "Philosophy in the flesh" by Lakoff and Johnson. A hefty work, but that was one of the publications that fundamentally changed my perspective on the human mind. The concept of embodied reasoning was eye-opening for me.

As I have an academic background in learning theory and developmental psychology, I'm pretty pessimistic about the current AI trend, autonomous driving etc. Most smart people in the field are chasing what are effectively more efficient regression functions for over 60 years now, and I almost never stumble upon approaches that have looked at what we know about actual human learning processes, development of the self etc.

Moravec's paradox[1] IMO should have been an inflection point for AI research. This is the level of problems AI research has to tackle if it ever wants to create AGI.

[1] https://en.wikipedia.org/wiki/Moravec%27s_paradox

Re: John Carmack: I’m going to work on artificial general intelligence

#896

Post by jacobush » Fri, Nov 15, 2019, 9:12 AM UTC

unityByFreedom wrote: post #552

jacobush wrote: post #534

Earlier quoted context omitted.

Ok, I'll take a shot: "it would follow that anyone tackling AGI would have some experience applying machine learning competitively in some public space". No, that would absolutely not follow. (I'm a pretty good devil's advocate, but I can't with this one.) And given AGI would come from some completely new breakthrough not related to the current practice of "machine learning", competitions may be completely moot. They…

Thank you for your reply. Turning a blind eye to existing knowledge may result in reinventing things that already exist. Nobody expects students to follow the same concepts as their teachers, the point is just to leverage existing knowledge. > I could even imagine a stumbling AGI being very stupid compared to just about any machine learning solution thrown at it - yet being undeniably AGI. Like a dog not being very g…

Another type from what? If you can't define it, how can you optimize for it? (Certainly not in the online competitions of today.)

Re: John Carmack: I’m going to work on artificial general intelligence

#897

Post by dkersten » Fri, Nov 15, 2019, 9:26 AM UTC

ghostbrainalpha wrote: post #869

dkersten wrote: post #680

Earlier quoted context omitted.

He also spreads misinformation. EDIT: Ok, I suppose I should back my claim up. Joe Rogan has pushed the “DMT is produced in our pineal gland” narrative, but there is no evidence to back this up. I’ll report a comment I made elsewhere and also link a separate reddit discussion which cites various sources. I will note that, in fairness to Joe, he said this a while ago, so perhaps he’s not so quick to jump the gun now,…

There is a difference between the current politicized phrase "spreading misinformation" and being wrong. Anyone who speaks on the record about their hobbies for thousands of hours will say some things that are incorrect. He might not understand something, and he is usually pretty humble about his knowledge level. But "spreading misinformation" is something that people do because they are intentionally misleading othe…

That’s fair, thanks for pointing it out. I’ll be more careful with how I express such things in future.

Regarding the pineal gland, it might be true, but it hasn’t been proven and multiple neuroscientists have stated that while DMT is similar to compounds found in the brain, it still functions quite differently and they have never seen any evidence to suggest that DMT exists in our bodies. There was a study finding it in mice brains, so it may still turn out that we have it in ours, but it’s definitely premature to make any such assumptions and definitely premature to repeat the trope.

Re: John Carmack: I’m going to work on artificial general intelligence

#898

Post by codingslave » Fri, Nov 15, 2019, 9:32 AM UTC

protomikron wrote: post #603

codingslave wrote: post #371

Earlier quoted context omitted.

I actually don't think mastering deep learning is very difficult. Theres a gazillion papers and ideas floating around, but the core concepts, that actually work, things like batch normalization, gradient descent, dropout, etc are all relatively simple. Most of the complexity comes from second rate scientists pushing their flawed research out into the public in some form of a status game

> [...] but the core concepts, that actually work, things like batch normalization, gradient descent, dropout, etc are all relatively simple. They may be simple, but it's controversial why they work. For example dropout is not really used much in recent CNN architectures, and it's just - I don't know - ~5 years old? So people don't even agree what the core concepts are ...

Sure, this is true. I just threw dropout in there without thinking much into it. The point is even if we include the techniques that have been replaced by newer ones, the total number of techniques is small. Also if youre learning deep learning for the first time, understanding why dropout was used, and then how batch normalization came to replace it is key to understanding neural networks. Same can be seen in network architectures, tracing the evolution of CNNs from VGG16 -> ResNet and why Resnet is better exposes one to the vanishing gradient problem, shows how the thought evolution happened, and gives hints to what could be next/builds intuition for the design of deep neural nets

Re: John Carmack: I’m going to work on artificial general intelligence

#899

Post by codingslave » Fri, Nov 15, 2019, 9:40 AM UTC

codetrotter wrote: post #423

codingslave wrote: post #371

Earlier quoted context omitted.

I actually don't think mastering deep learning is very difficult. Theres a gazillion papers and ideas floating around, but the core concepts, that actually work, things like batch normalization, gradient descent, dropout, etc are all relatively simple. Most of the complexity comes from second rate scientists pushing their flawed research out into the public in some form of a status game

For anyone unfamiliar with all but the most trivial details, do you have some good papers to recommend, to save us from wading through all the rest?

Get some basics of linear algebra down. Eigenvectors, Eigenvalues. Nail down Matrix Factorization, Principal Components, and the relationship between the two.

Learn softmax, logit function, different activation functions. When to use them. Difference between classification, binary classification, multi label prediction etc. Theyre all similar, just use a few different functions in the neural net

After this, go through some optimization theory and learn the different algorithms for optimizing neural nets, i.e. Adam vs RMSProp.

Then I would just get a list of all the top network architectures, then go through their white papers. Do this chronologically. Start at ~2012. Basically all the network architectures build on each other. So take the first good working deep CNN (alexnet), find out why it worked. Then move to VGG, why did that one work? What problems were solved? then move onwards.

^Do this for computer vision, then again for NLP (Word Vectors) and transformers (BERT, XLNet, etc).

Then youre done.

Theres also GANs etc, but that stuff is extra.

From there, choose whatever specialty you wanna research, and just grab the state of the art.

Re: John Carmack: I’m going to work on artificial general intelligence

#900

Post by marcus_holmes » Fri, Nov 15, 2019, 9:44 AM UTC

simonh wrote: post #894

marcus_holmes wrote: post #715

Earlier quoted context omitted.

they'd do it like we do: by comparing the new situation to previous ones we know about, and applying the model that fits best, and then adapting to results. For humans, the more previous ones we know about, the better, because we have more chance of applying a model that works in the new environment. That's called "experience".

That’s a very broad, general description of behaviour that doesn’t actually describe an implementation. In fact it could apply to many completely different possible implementations. I suspect though that humans do more than this, that we have a way of either constructing entirely new models from scratch, or of dramatically adapting models to new situations without mere iterative fitting to feedback. Humans are actual…

I don't know. I'm not so sure that we can create new working models from scratch. We definitely learn by iterative feedback: babies wiggle stuff and watch what happens to learn how to move their bodies. Learning to ride a bicycle is mostly about falling off bicycles until you learn how not to.

I've seen people apply their normal behaviour to situations that have changed, and then get totally confused (and angry) as to why the result isn't the same. Observe anyone travelling in a new country for examples ("why don't they show the price with the sales tax included here? This is ridiculous!").

In a perfect world, sure, we'd construct a rational mental model of a new situation and test it carefully to ensure it matched reality before trusting it, and then apply it correctly to the new situation. But it's not a perfect world, and people don't actually do that. Usually we charge in and then cope with the results.

Of course, I'm not saying that AI should do that. It'll be interesting to see how a "good" general AI copes with a genuinely new situation.