Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack: I’m going to work on artificial general intelligence

facebook.com

Post reply on HN
Print view View original

671–680 of 930 posts

Re: John Carmack: I’m going to work on artificial general intelligence

#671

Post by taneq »

snowwrestler wrote: post #426
nradov wrote: post #187

Earlier quoted context omitted.

Current ML technology probably has little or nothing to do with whatever technology will eventually be needed to produce true AGI.

As I like to say: lots of people are working on making a car that is smart enough to drive itself wherever a human wants to go. How many people are working on a car smart enough to tell humans to fuck off, it doesn’t feel like driving anywhere today?

Why would you want that, though? What we really want from AGI is mostly just things that are smart enough to 'do what I mean' but dumb enough to not mind being slaves.

Re: John Carmack: I’m going to work on artificial general intelligence

#672

Post by someguyorother »

Jach wrote: post #257
mrfusion wrote: post #174

Earlier quoted context omitted.

I don’t think there are many researchers in AGI. AFAIK it’s kind of a joke field because no one has any clue how to approach true AGI. Please correct me if I’m wrong.

People have approaches. There's no end to half-assed "I thought about this for 10 seconds, how hard could it be!" solutions, really old approaches from decades ago where the brightest academics thought they could lick the problem over a summer, and some new public or hidden approaches that might be promising but (I can't know of course) I predict will still look a lot different than the final thing. I think a big rea…

MIRI's concerns are vastly overrated IMHO. Any AGI that's intelligent enough to misinterpret its goals to mean "destroy humanity" is also intelligent enough to wirehead itself. Since wireheading is easier than destroying humanity, it's unlikely that AGI will destroy humanity.

Trying to make the AGI's sensors wirehead-proof is the exact same problem as trying to make the AGI's objective function align properly with human desires. In both cases, it's a matter of either limiting or outsmarting an intelligence that's (presumably) going to become much more intelligent than humans.

Hutter wrote some papers on avoiding the wireheading problem, and other people have written papers on making the AGI learn values itself so that it won't be tempted to wirehead. I wouldn't be surprised if both also mitigate the alignment problem, due to the equivalence between the two.

Re: John Carmack: I’m going to work on artificial general intelligence

#673

Post by gray_-_wolf »

0-_-0 wrote: post #577
fastball wrote: post #535

Earlier quoted context omitted.

Sooooo, The Talos Principle? https://en.wikipedia.org/wiki/The_Talos_Principle

Puzzle game with a great story. I recommend it to the HN people.

I loved it, but my issue with that game was severe motion sickness after 20-30 minutes... never finished it :(

Re: John Carmack: I’m going to work on artificial general intelligence

#674

Post by jacquesm »

MaximumYComb wrote: post #579
Animats wrote: post #437

Earlier quoted context omitted.

I've long taken the position that intelligence is mostly about getting through the next 10-30 seconds of life without screwing up. Not falling down, not running into stuff, not getting hurt, not breaking things, making some progress on the current task. Common sense. Most of animal brains, and a large fraction of the human brain, is devoted to managing that. On top of that is some kind of coarse planner giving goals…

I was thinking about the manipulation issue tonight. I'd been throwing a tennis ball in the pool with my kids and I realised how instinctual my ability to catch was. A ball leaves my kids hands and I move my hand to a position, fingers just wide enough for a ball, and catch it. All of it happens in a fraction of a second. The human brain can model the physics of a ball in flight, accurately, and quickly. As the ball…

Dogs can do this too. And quite a bit more impressive than most humans.

Re: John Carmack: I’m going to work on artificial general intelligence

#675

Post by simonh »

goldenkey wrote: post #663
simonh wrote: post #657

Earlier quoted context omitted.

Alphago and it's kind are doing some things that we do, for sure. We do utilise pattern recognition and some of the neurological tools we bring to bear on these problems might look a bit like Alphago. The thing is those are parts of our neurology that have little to do with general intelligence. I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level genera…

I can't agree with you. I know many chess players at master and Grand Master level. Look at Bobby Fischer too. Human specialization does not carry over very well to other tasks, only marginally...

I don't think that's a disagreement, I think you're right. Most of the benefit someone like that would get from their competence in Chess or Go would be incidental ones. In fact I would say your experience confirms my understanding of this, optimizing for a single domain in the way Alphago does or even in the way humans do, has little to do with general intelligence.

Re: John Carmack: I’m going to work on artificial general intelligence

#676

Post by geogra4 »

Jaruzel wrote: post #518

Carmack's post in full: Starting this week, I’m moving to a "Consulting CTO” position with Oculus. I will still have a voice in the development work, but it will only be consuming a modest slice of my time. As for what I am going to be doing with the rest of my time: When I think back over everything I have done across games, aerospace, and VR, I have always felt that I had at least a vague “line of sight” to the sol…

Too bad he didn't go for his runner up. cost effective/mass produceable fission reactors could save humanity.

Re: John Carmack: I’m going to work on artificial general intelligence

#677

Post by xamuel »

sytelus wrote: post #412
xamuel wrote: post #319

I've been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I'll bet Carmack can offer some significant contributions. I'll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field. Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward…

Could you summarize what exactly interesting insights did you gained by casting this problem as the election system? Unfortunately abstract fails to communicate this (I hate "teaser-only" abstracts!). Also, it might be easy to go from relative metric to sort of absolute by comparing against a "starndard" agent, for example, a random agent.

The #1 thing is it led to an elegant notion of how to compare intelligence (or rather, a parametrized family of intelligence comparators). There's a theorem called Arrow's Theorem, which basically says there's no good way to decide elections between more than 2 candidates. There are a handful of requirements an election-deciding method should have, and no method satisfies all those requirements. But Arrow's theorem has a loophole, discovered in the 1970s: if there are infinitely many voters, then there are methods of deciding the election which satisfy all Arrow's requirements. Economists in the 1970s exactly characterized these methods, in terms of a mathematical device called an "ultrafilter". Taking their characterization, a definition for relative intelligence is immediate--it writes itself.

If you're already familiar with ultrafilters, then the relative intelligence definition is SO elegant you're like, "Whoah. How can a definition of relative intelligence be that simple?" Of course, if you're not familiar with ultrafilters, then the definition is just as complicated as the definition of ultrafilters, which is quite complicated. So the definition of relative intelligence is like a simple computer program which imports from a complicated library in order to achieve that simplicity.

Re: John Carmack: I’m going to work on artificial general intelligence

#678

Post by xamuel »

omalleyt wrote: post #576
xamuel wrote: post #319

I've been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I'll bet Carmack can offer some significant contributions. I'll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field. Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward…

Unfortunately, your intelligent agents qualify as optimization algorithms and therefore the No Free Lunch Theorem applies: https://ti.arc.nasa.gov/m/profile/dhw/papers/78.pdf I.e. across the space of all possible environments, all agents perform equally well

As others pointed out, the NFLT only applies if the environments are uniformly distributed. In the paper, they are not uniformly distributed.

Re: John Carmack: I’m going to work on artificial general intelligence

#680

Post by dkersten »

MetalGuru wrote: post #451
copperx wrote: post #442

Earlier quoted context omitted.

Why do people of such intelligence subject themselves to being interviewed by dumb-as-a-rock Joe Rogan? I must admit that I often watch his interviews because he invites interesting people, but I can't help but cringe when Rogan gives his opinions.

Seriously. That guy is a stoned idiot who massively overestimates the insight of his high ramblings.

He also spreads misinformation.

EDIT: Ok, I suppose I should back my claim up.

Joe Rogan has pushed the “DMT is produced in our pineal gland” narrative, but there is no evidence to back this up. I’ll report a comment I made elsewhere and also link a separate reddit discussion which cites various sources. I will note that, in fairness to Joe, he said this a while ago, so perhaps he’s not so quick to jump the gun now, I don’t know, I don’t listen to his podcasts, but perhaps he’s better now.

“We all have it in our bodies” — This is an often repeated myth that has never been proven. The myth originates from Rick Strassman’s work, who himself has said that he only detected a precursor, not DMT itself and that everything else he wrote about it was hypothetical speculation. There have, apparently, been recent studies that found DMT synthesised in rat brains, but it has not yet been proven whether this translates to humans or not. Cognitive neuroscientist Dr. Indre Viskontas stated that while DMT shares a similar molecular structure to seritonin and melatonin, there is no evidence that it is made inside the brain. Similarly, Dr. Bryan Yamamoto of the neurosciencedepartment at the University of Toledo said: “I know of no evidence that DMT is produced anywhere in the body. It’s chemical structure us similar to serotonin and melatonin, but their endogenous actions are very different from DMT.”

This reddit discussion also links various sources, although I didn’t check them all myself: https://www.reddit.com/r/JoeRogan/comments/mwz2h/dmt_has_nev...

Post reply on HN