Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

181–190 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#181

Post by dudouble »

lagrange77 wrote: post #148
paxys wrote: post #131

Earlier quoted context omitted.

We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.

From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.

Thank you for this comment. I'd never really considered this and it is blowing my mind.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#182

Post by HL33tibCe7 »

semi-extrinsic wrote: post #139
Tepix wrote: post #7

I wonder if Carmacks moral compass is in order. First he sticks around at Facebook, now he endangers humanity with AGI. And i'm only half joking.

The meme that AGI, if we ever have it, will somehow endanger humanity is just stupid to me. For one, the previous US president is the perfect illustration that intelligence is neither sufficient nor necessary for gaining power in this world. And we do in fact live in a world where the upper echelons of power mostly interact in the decidedly analog spaces of leadership summits, high-end restaurants, golf courses and c…

It sounds like you haven’t really thought through AI safety in any real detail at all. Airgapping and the necessity of human input are absolutely not ways to prevent an AGI gaining access to a system. A true, superintelligent AGI could easily extort (or persuade) those humans.

If you think concerns over AGI are “stupid”, you haven’t thought about it enough. It’s a massive display of ignorance.

The Computerphile AI safety videos are an approachable introduction to this topic.

Edit: just as one very simple example, can you even imagine the destruction that could (probably will) occur if (when) a superintelligent AGI gets access to the internet? Imagine the zero days it could discover and exploit, for whatever purpose it felt necessary. And this is just the tip of the iceberg, just one example off the top of my head of something that would almost inevitably be a complete catastrophe.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#183

Post by blibble »

Arcuru wrote: post #118
ge96 wrote: post #86

Earlier quoted context omitted.

I wonder about this, if you had great/true automation, free energy from the sun, is there any need to do anything. As in value of money.

If you want to look into it more, that situation is usually called a post-scarcity economy[1]. It's talked about and depicted in a few fictionalized places, including Star Trek. [1] - https://en.wikipedia.org/wiki/Post-scarcity_economy

in Star Trek: the Federation has unlimited energy and the ability to replicate most forms of matter

but human(oid) intelligence is still scarce, and they don't have AGI (other than Data)

there is however a society that has no need for humanoid intelligence, and that's the Dominion

and I suspect that is what our society would turn into if AGI is invented (and not the Federation)

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#184

Post by ryanSrich »

paxys wrote: post #131
Jack000 wrote: post #116

I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…

We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.

> AGI

Idk what prompted you to say this, but is there a version of AGI that isn't "real" AGI? I don't know how anyone could fake it. I think marketing departments might say whatever they want, but I don't see any true engineers falling for something masquerading as AGI.

If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#185

Post by danbmil99 »

cgrealy wrote: post #130

I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet. The idea that we can control something like that is laughable.

Why is it so important to you that humans be the smartest beings on the planet?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#186

Post by rebelos »

madrox wrote: post #66
Trasmatta wrote: post #2

Interesting to see how he's progressed with this. When he first announced he was getting into AI it sounded almost like a semi retirement thing: something that interested him that he could do for fun and solo, without the expectation that it would go anywhere. But now he seems truly serious about it. Wonder if he's started hiring yet.

I got the same impression, and maybe it still is. You can still raise money for a retirement project if the goal of the money is to hire a staff. VC money isn't solely for young 20-something founders who want to live their job.

Carmack sounds like someone who lives his job, so I don't think age/life stage is a factor here.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#187

Post by gharperx »

WalterBright wrote: post #153

It will decide our fate in a microsecond: extermination.

I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world.

I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#188

Post by MichaelCollins »

colinmhayes wrote: post #168
paxys wrote: post #131

Earlier quoted context omitted.

We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.

There’s certainly the philosophy side of AGI, but there’s also the practical side. Does the Chinese room understand Chinese? If your goal is just to create a room that passes Chinese Turing tests that doesn’t matter.

The philosophy side of the matter seems meaningless, it interrogates the meaning of language, not the capabilities of technology. When people ask "Could machines think?" the question isn't really about machines, it's about precisely what we mean by the word 'think'.

Can a submarine swim? Who cares! What's important is that a submarine can do what a submarine does. Whether or not the action of a submarine fits the meaning of the word 'swim' should be irrelevant to anybody except poets.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#189

Post by konschubert »

paxys wrote: post #131
Jack000 wrote: post #116

I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…

We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.

Sentience is ill-defined and therefore doesn't exist.
Post reply on HN