Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

311–320 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#311

Post by semi-extrinsic »

sabellito wrote: post #200
semi-extrinsic wrote: post #139

Earlier quoted context omitted.

The meme that AGI, if we ever have it, will somehow endanger humanity is just stupid to me. For one, the previous US president is the perfect illustration that intelligence is neither sufficient nor necessary for gaining power in this world. And we do in fact live in a world where the upper echelons of power mostly interact in the decidedly analog spaces of leadership summits, high-end restaurants, golf courses and c…

There's a wondeful youtube channel from a researcher who focusses exactly on this topic, I think you should check it out: https://www.youtube.com/watch?v=ZeecOKBus3Q

I watched the whole thing. Man spent a lot of breath asserting that an AGI will have broadly the same types of goals that humans do. Said exactly zero words about why we won't just be able to tell the AGI "no, you're not getting what you want", and then turn it off.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#314

Post by MichaelCollins »

adamsmith143 wrote: post #304
MichaelCollins wrote: post #173

Earlier quoted context omitted.

There is an inverse relationship between the age of a futurist and the amount of time they think it will take for their predictions to become true. In other words, people making these sort of predictions about the future are biased towards believing they'll be alive to benefit from it.

> There is an inverse relationship between the age of a futurist and the amount of time they think it will take for their predictions to become true. I think calling Carmack a Futurist is pretty insulting.

Why? Because he also wrote some game engines?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#315

Post by rychco »

dqpb wrote: post #243
mellosouls wrote: post #87

I am not saying the intention here is the same, but the headline doesn't inspire confidence. There's something incredibly creepy and immoral about the rush to create then commercialise sentient beings. Let's not beat about the bush - we are basically talking slavery. Every "ethical" discussion on the matter has been about protecting humans, and none of it about protecting the beings we are in a rush to bring into lif…

If AGI is possible, it’s immoral not to create it.

How so? It’s not immoral to abstain from creating life (having children, biologically speaking). Am I missing something?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#316

Post by makeitdouble »

wcerfgba wrote: post #239

Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentall…

I see that as two separate goals.

One is to build something inteligent (an AGI), and the other is something human like. Intuitively we could hit the AGI goal first and aim for human like after that if we feel like it.

In the past, human like inteligent seemed more approachable for I think mostly emotional reasons, but for our current trajectory if we get anything inteligent we’d still have reach a huge milestone IMO.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#317

Post by PaulDavisThe1st »

madrox wrote: post #230
otikik wrote: post #222

I'm slightly scared that they'll succeed. But not in the usual "robots will kill us" way. What I am afraid of is that they succeed, but it turns out similar to VR: as an inconsequential gimmick. That they use their AGIs to serve more customized ads to people, and that's where it ends.

If a different person were doing it, I think that'd be fair, but Carmack has a track record of quality engineering and quality product. I don't think you can blame Quest on him given the way he chose to exit.

whether or not something is or is not an inconsequential gimmick doesn't have much to do with quality engineering.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#318

Post by gambiting »

HL33tibCe7 wrote: post #291
gambiting wrote: post #257

Earlier quoted context omitted.

>> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn…

> Like....just pull the plug. Watch this video https://youtu.be/3TYT1QfdfsM

It's midnight, so I'm not super keen on watching the whole thing(I'll get back to it this weekend) - but the first 7 minutes sounds like his argument is that if you build a humanoid robot with a stop button, the robot will fight you to prevent you pressing its own stop button if given an AGI? As if the very first instance of AGI is going to be humanoid robots that have physical means of preventing you from pressing their own stop button?

Let me get this straight - this is an actual, real, serious argument that they are making?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#319

Post by luma »

semi-extrinsic wrote: post #250
HL33tibCe7 wrote: post #182

Earlier quoted context omitted.

It sounds like you haven’t really thought through AI safety in any real detail at all. Airgapping and the necessity of human input are absolutely not ways to prevent an AGI gaining access to a system. A true, superintelligent AGI could easily extort (or persuade) those humans. If you think concerns over AGI are “stupid”, you haven’t thought about it enough. It’s a massive display of ignorance. The Computerphile AI sa…

So what if the AGI discovers a bunch of zero days on the internet? We can just turn the entire internet off for a week and be just fine, remember? And exactly how does the AGI extort or persuade humans? What can it say to me that you can't say to me right now?

Sends you a text from your spouse's phone number that an emergency has happened and you need to go to xyz location right now. Someone else is a gun owner and they get a similar text, but their spouse is being held captive and are sent to the same location with a description of you as the kidnapper. Scale this as desired.

Use your imagination!

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#320

Post by cgrealy »

pgcj_poster wrote: post #252
cgrealy wrote: post #130

I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet. The idea that we can control something like that is laughable.

> AGI means humans are no longer the smartest entities on the planet. Superintelligence and AGI are not the same thing. An AI as smart as an average 5 year old human is still an Artificial General Intelligence.

I don’t buy that. The gap between “no AGI” and “AGI but it’s a child” is orders of magnitude greater than the gap between “5 year old” and “smartest human”.
Post reply on HN