John Carmack: I’m going to work on artificial general intelligence • Hacker News Board

Re: John Carmack: I’m going to work on artificial general intelligence

#861

Post by tomxor » Thu, Nov 14, 2019, 8:42 PM UTC

tomxor wrote: post #763

shashankp wrote: post #696

Earlier quoted context omitted.

Carmack creates AI, AI is used to invent effective fission reactors. Check and mate.

- fusion reactors then used to power spaceX ships to go to Mars... wait what! As long as his next project is not teleportation gateways i guess we are safe. [edit] it's also 2019 :D > In the year 2019, the player character (an unnamed space marine) has been punitively posted to Mars after assaulting a superior officer, who ordered his unit to fire on civilians. The space marines act as security for the Union Aerospac…

how come 90% of the time HN readers can't see the funny side in anything.

Re: John Carmack: I’m going to work on artificial general intelligence

#862

Post by tim333 » Thu, Nov 14, 2019, 9:29 PM UTC

mrfusion wrote: post #174

echelon wrote: post #122

Earlier quoted context omitted.

What a weird future it would be if Carmack turns out to be the one to figure out the critical path and get it all working. An entire field of brilliant researchers be damned. History books (for as long as those continue to exist) would cite AGI as his major contribution to society, and his name would be more renowned than Edison or Tesla. An Einstein. None of his other contributions will matter, as the machines will…

I don’t think there are many researchers in AGI. AFAIK it’s kind of a joke field because no one has any clue how to approach true AGI. Please correct me if I’m wrong.

Wikipedia:

>A 2017 survey of AGI categorized forty-five known "active R&D projects" that explicitly or implicitly (through published research) research AGI, with the largest three being DeepMind, the Human Brain Project, and OpenAI.

Hassabis and DeepMind have a fairly organised approach of looking at how real brains work and trying to model different problems like Atari games then Go and recently Starcraft. Not quite sure what's next up.

Re: John Carmack: I’m going to work on artificial general intelligence

#863

Post by marcus_holmes » Thu, Nov 14, 2019, 9:32 PM UTC

BjorksEgo wrote: post #806

bumby wrote: post #787

Earlier quoted context omitted.

This may be a clunky analogy, but is this fundamentally different than a killing a human as long as we keep a record of their DNA sequence? Maintaining the information doesn't seem enough to negate snuffing out the execution of that information

Not if we persist the state of the mind before we turn them off. Can't do that with humans

yet

Re: John Carmack: I’m going to work on artificial general intelligence

#864

Post by DoctorOetker » Thu, Nov 14, 2019, 9:32 PM UTC

sailingparrot wrote: post #219

DoctorOetker wrote: post #153

Earlier quoted context omitted.

the recent result on 15% optimal learning error rate for binary classification, could have been derived with pencil on paper by anyone...

Sure, anyone can also do manual backpropagation of a very small modern neural network with pencil and paper, and yet it took decades to be there. Everything is easy in hindsight. And that's without mentioning that the specific result you are talking about is an infinitesimal progress when compared to "AGI".

the GP stated:

> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.

So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory".

Why would you compare this with performing manual backpropagation?

>Everything is easy in hindsight

Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation...

The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.

Re: John Carmack: I’m going to work on artificial general intelligence

#865

Post by strbean » Thu, Nov 14, 2019, 9:34 PM UTC

the_af wrote: post #665

Animats wrote: post #313

This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. They have to operate in a world, interact with others, survive, and accomplish goals. Simulator technology is now good enough that you can do quite realistic worlds. Imagine The Sims, with a lot more internal smarts and real physics, as a base for work. Robotics has the same issue…

> This is encouraging. If you're going to work on artificial general intelligence, a reasonable context in which to work on it is game NPCs. I don't think so. Game NPCs don't need AI, which would be way overkill; they just need to provide the illusion of agency. I think for general AI you need a field where any other option else would be suboptimal or inadequate, but in videogames general AI is the suboptimal option.…

I don't think parent was referring to game NPCs as a reasonable application of AGI, but rather as a reasonable domain conducive to the development of AGI.

Re: John Carmack: I’m going to work on artificial general intelligence

#866

Post by marcus_holmes » Thu, Nov 14, 2019, 9:37 PM UTC

bumby wrote: post #787

marcus_holmes wrote: post #719

Earlier quoted context omitted.

well, "kill" becomes moot if the code is preserved. Like "killing" another player in a multi-player game. You're not actually killing them.

This may be a clunky analogy, but is this fundamentally different than a killing a human as long as we keep a record of their DNA sequence? Maintaining the information doesn't seem enough to negate snuffing out the execution of that information

The generic AI could be "playing" in a thousand virtual environments at once. Killing one of them doesn't really have a parallel in human life, or ethics.

I mean, yes, you killed a sentient being. But if that sentient being has a thousand concurrent lives, then what does "killing" one of those lives even mean? And if it can respawn another identical life in a millisecond, does it even count as killing?

I suspect that having sentient virtual entities will provide philosophy and ethics majors a lot of deep thinking room. As it already has for SciFi authors.

Re: John Carmack: I’m going to work on artificial general intelligence

#867

Post by joe_the_user » Thu, Nov 14, 2019, 9:41 PM UTC

ivanhoe wrote: post #559

PeterStuer wrote: post #475

Earlier quoted context omitted.

I did my msc thesis in AI back then writing a dedicated simulater for a specific robot used in autonomous systems research. You find that especially when trying to faithfully reproduce sensor signals you need to dive deep into not just the physics of e.g. infrared light, but also the specific electronic operation of the sensor itself. But that kind of realism is not needed for all AGI research. I also spent some year…

Modeling everything realistically is super hard - any interaction with the real-world is so full of the weirdest unexpected electric and mechanical issues. Who hasn't tried it first-hand can't imagine the half of ways that will almost certainly go wrong on the first try :) ... but as you've said for developing the AGI as a concept simplified worlds should work just fine.

Indeed, I don't think humans themselves model the world realistically, I think they model the world close enough but are able to adapt their prediction process for situation when their prediction don't work and/or their knowledge is inadequate. To model such a "satisficing" process, you don't need exact simulation.

Re: John Carmack: I’m going to work on artificial general intelligence

#868

Post by quicknir » Thu, Nov 14, 2019, 9:48 PM UTC

criddell wrote: post #854

quicknir wrote: post #799

Earlier quoted context omitted.

I normally don't bother but this comment is so profoundly ridiculous I had to say something. Tenured ML professors at the top 100 or so universities in the world aren't "most of us". A very large chunk of these people are geniuses. Those jobs are incredibly hard to get, and most of these people are reading everything that is getting published, on an ongoing basis, and are outputting something novel, on an ongoing bas…

> Carmack doing it would just be another drop in the bucket. If this research is as compute intensive as it seems to be, Carmack's contribution might be that he increases the rate other researchers can add their drops to the bucket. Carmack isn't the first techie to take on a big hard problem. Jeff Hawkins, a name many of us also know, did as well.

Yes, he may well improve some algorithm, or rewrite some commonly used tool to improve efficiency. And researchers are often not incentivized to do that, so it would be great. But a far cry from the picture people are painting about him soaking up the field and using his genius to solve some major problem quickly.

If by "techie" you mean, professional software engineer, that's fine, but there's no reason to assume that a professional software engineer is going to be magically better at AI research than... professional AI researchers? He's probably going to be substantially worse.

Also, your statement below:

> That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.

Makes it clear to me that you don't really get it. Carmack, at best, might know enough right now to be in a PhD program. I doubt that he has anywhere near as much knowledge, insight, or ideas for research, as top graduate students. He's in no position to mentor graduate students.

Re: John Carmack: I’m going to work on artificial general intelligence

#869

Post by ghostbrainalpha » Thu, Nov 14, 2019, 10:18 PM UTC

dkersten wrote: post #680

MetalGuru wrote: post #451

Earlier quoted context omitted.

Seriously. That guy is a stoned idiot who massively overestimates the insight of his high ramblings.

He also spreads misinformation. EDIT: Ok, I suppose I should back my claim up. Joe Rogan has pushed the “DMT is produced in our pineal gland” narrative, but there is no evidence to back this up. I’ll report a comment I made elsewhere and also link a separate reddit discussion which cites various sources. I will note that, in fairness to Joe, he said this a while ago, so perhaps he’s not so quick to jump the gun now,…

There is a difference between the current politicized phrase "spreading misinformation" and being wrong.

Anyone who speaks on the record about their hobbies for thousands of hours will say some things that are incorrect. He might not understand something, and he is usually pretty humble about his knowledge level.

But "spreading misinformation" is something that people do because they are intentionally misleading others, or have something to gain.

I don't think he is benefiting much from the pineal gland narrative. And it sounds like from the information you cited, it may even be correct, even if its premature to state it as fact.

Re: John Carmack: I’m going to work on artificial general intelligence

#870

Post by whamlastxmas » Thu, Nov 14, 2019, 10:27 PM UTC

goatlover wrote: post #222

whamlastxmas wrote: post #66

Earlier quoted context omitted.

What is your source for saying AGI will probably wipe out humanity? How could we ever even attempt to guess at the motivations of something we can barely comprehend and doesn't even exist yet?

The main concern is not that it's like Skynet and wishes us harm, but that it does things harmful to us because the means to accomplish the AI's goals are at odds with human values, which was unanticipated by the human creators, since the AGI is coming up with its own solutions. And the AGI doesn't have the same values as humans, so it doesn't care if its solutions are harmful.

I guess the risk seems more real if it's a question of "does it share human values". Which is possibly not a one in a million chance it doesn't - it could be a 50/50 chance or worse.