Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

421–430 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#421

Post by semi-extrinsic »

dinosaurdynasty wrote: post #373
semi-extrinsic wrote: post #311

Earlier quoted context omitted.

I watched the whole thing. Man spent a lot of breath asserting that an AGI will have broadly the same types of goals that humans do. Said exactly zero words about why we won't just be able to tell the AGI "no, you're not getting what you want", and then turn it off.

He covers that in another video: https://www.youtube.com/watch?v=3TYT1QfdfsM

Oh FFS. He starts off by giving the AGI a robot body that can kill things (don't do that in the first place!), and then he adds a stop button in a type of location that wouldn't pass muster on a table saw, let alone an industrial robot.

If you've placed the stop button such that you cannot access it safely in all possible failure modes of the dangerous machine, be it bench grinder or an AGI or whatever, you have failed the very basics of industrial safety and you must go home and be sad, for you will not be receiving a cookie.

And the thought experiments of this guy are completely incoherent. At the same time the robot has human-level (or greater) intelligence, yet is extremely stupid, and you can somehow program it in an extremely simplistic way.

Sorry, but all you've convinced me of is that this is just a random guy on YouTube with no particular education of relevance nor other qualifications (I checked), who is very pleased about his own ideas and is quite lacking in the ability to think critically.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#422

Post by version_five »

labrador wrote: post #419
godelski wrote: post #371

Earlier quoted context omitted.

> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.

> gullibility gap Search turns up very little except a book by a Christian author. Could you define what you mean by gullibility gap?

It's coined in context.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#423

Post by darawk »

godelski wrote: post #412
sigmoid10 wrote: post #403

Earlier quoted context omitted.

As an AI researcher myself I can definitely see why people used to have these sentiments, but I also have to point out how the big papers of the last ~12 months or so have changed the landscape. Especially multi-modal models like Gato. To my surprise, a lot of people (even inside the community) still tend to put human intelligence on some sort of pedestal, but I believe once a system (or host of systems) achieves a s…

I'm not sure Gato actually demonstrates intelligence. It definitely does demonstrate the ability to perform a wide variety of tasks, but I don't think that is actually intelligence. Gato has yet to demonstrate that it understands things. I do think it is easy to fall prone to the gullibility gap (and also take too strong of an opposing view, which I may be victim to).

How are you defining "understands things"? Or more precisely, what are the observable consequences of understanding things that you are waiting to see?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#424

Post by celdon25 »

mhh__ wrote: post #275
Trasmatta wrote: post #2

Interesting to see how he's progressed with this. When he first announced he was getting into AI it sounded almost like a semi retirement thing: something that interested him that he could do for fun and solo, without the expectation that it would go anywhere. But now he seems truly serious about it. Wonder if he's started hiring yet.

Does he have the expertise to pull it off as an individual?

I would say so. Most AGI work so far has been pursued by academics with a lack of software engineering best practices. Codebases get bloated and hard to understand. Ramp-up time measured in years. This hinders experimentation, which you have to optimize for with projects like this.

I would suspect Carmack understands this, and also understands what resources to consult, and how to spot the bullshit artists in the field.

I'd work for him in a heartbeat.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#425

Post by nightski »

Does anyone have any good links/podcasts/books on content that explores what is AGI and how to define it? Probably the best stuff I've listened to so far are Fridman podcasts with guests like Jeff Hawkins or Joscha Boch. But I'd love to read a book that explores this topic if any even exist.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#426

Post by hoherd »

nightski wrote: post #425

Does anyone have any good links/podcasts/books on content that explores what is AGI and how to define it? Probably the best stuff I've listened to so far are Fridman podcasts with guests like Jeff Hawkins or Joscha Boch. But I'd love to read a book that explores this topic if any even exist.

Life 3.0 by Max Tegmark is great. It reads like a text book. It's several years old though and I wonder how much has changed since it was written.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#427

Post by darawk »

wcerfgba wrote: post #239

Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentall…

I think the burden of argumentation is on the person proposing the constraint. I can only think of one argument for why embodiment might be necessary, and that is our only existence proofs for AGI (humans, maybe some animals) are embodied. But those existence proofs also share a bunch of other properties that probably aren't particularly relevant to AGI. Being carbon-based, having DNA, breathing oxygen, etc.

Why do you think embodiment should be necessary?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#428

Post by dinosaurdynasty »

smnplk wrote: post #208

I think its a silly idea that consciousness can be produced by computation.

Humans are made entirely out of quantum fields. I don't think you can get out of the idea that human consciousness isn't produced by computation without positing something like a non quantum field theory soul, for which there is no evidence.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#429

Post by semi-extrinsic »

blibble wrote: post #406
semi-extrinsic wrote: post #328

Earlier quoted context omitted.

Sorry, I just wasted ten minutes watching a YouTube video that someone else in this thread asserted was going to explain everything. Said video was entirely tangential to the point in question - why can't we just tell the AGI to sod off with it's wishes and goals. At the end of the day it is all just bits going through wires, which we are perfectly capable of disconnecting. Now I'm not going to spend a couple of hour…

> Now I'm not going to spend a couple of hours watching a film that has a mediocre IMDB rating. your loss > For the AGI to actually accomplish something of real-world relevance, us humans have to agree. Donald Trump managed to get control of the US nuclear arsenal

Yes, now we are getting somewhere. Donald Trump managed to somehow get control in a very limited sense of the US nuclear arsenal.

First of all, how did he get this control? Certainly it was not via his profound intelligence, but because other humans chose to give him control.

Second of all, we know beyond a shadow of a doubt, that if Donald Trump had ordered a nuclear strike in a situation where it was not consistent with US nuclear weapons doctrine, his command would have been disobeyed and the 25th amendment would have been enacted poste haste.

So again, how would an AGI be able to get into office or into another position of power? And how would it be able to order a nuclear strike in a situation inconsistent with nuclear doctrine and not be overriden by those lower in the chain of command? Why would there be different rules applying to the AGI as compared to the human?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#430

Post by mkl »

ryanSrich wrote: post #229
danielheath wrote: post #209

Earlier quoted context omitted.

What’s it going to do, break out of its own simulation?

Real AGI would adapt and fool a human into letting it out. Or escaping through some other means. That's the entire issue with AGI. Once it can learn on its own there's no way to control it. Building in fail safes wouldn't work on true AGI, as the AGI can learn 1000x faster than us, and would free itself. This is why real AGI is likely very far away, and anything calling itself AGI without the ability to learn and ada…

That doesn't seem to make sense. There's no reason to think an "AGI can learn 1000x faster than us", unless that's your idiosyncratic definition of "real AGI". Something as smart and capable as the average human would certainly be real AGI by everyone else's definition.
Post reply on HN