Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

Post reply on HN
Print view View original

211–220 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#211

Post by phatfish »

falcrist wrote: post #76
mushufasa wrote: post #52

Earlier quoted context omitted.

I think he very clearly has an amoral attitude towards technology development -- "you can't stop progress." He does describe his own "hacker ethic" and whatever he develops he may make more open than OpenAI. though I think he has some moral compass around what he believes people should do or not do with technology. For example, he has publicly expressed admiration for electric cars / cleantech and SpaceX's decision t…

He also has a vastly different take on privacy than most of us seem to have. He thinks it'll eventually go away and it won't be bad when it does. I believe he talked about it in one of his quakecon keynotes. As a LONG time admirer of Carmack (I got my EE degree and went into embedded systems and firmware design due in no small part to his influence), I feel like he's honest and forthright about his stances, but also…

He is a workaholic, in a positive way. But i get the feeling as long as he has a problem he enjoys "grinding out" the solution to, not much else matters -- apart from the obvious of family and close friends.

Still, I can't fault his honesty. He doesn't seem to hold anything back in the interviews I've seen.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#212

Post by lagrange77 »

dudouble wrote: post #181
lagrange77 wrote: post #148

Earlier quoted context omitted.

From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.

Thank you for this comment. I'd never really considered this and it is blowing my mind.

I'm no expert, take it with a grain of salt :)

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#213

Post by treesprite82 »

lagrange77 wrote: post #148
paxys wrote: post #131

Earlier quoted context omitted.

We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.

From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.

> capable of constructing a slightly better version of itself

With just self-improvement I think you hit diminishing returns, rather than an exponential explosion.

Say on the first pass it cleans up a bunch of low-hanging inefficiencies and improves itself 30%. Then on the second pass it has slightly more capacity to think with, but it also already did everything that was possible with the first 100% capacity - maybe it squeezes out another 5% or so improvement of itself.

Similar is already the case with chip design. Algorithms to design chips can then be ran on those improved chips, but this on its own doesn't give exponential growth.

To get around diminishing returns there has to be progress on many fronts. That'd mean negotiating DRC mining contracts, expediting construction of chip production factories, making breakthroughs in nanophysics, etc.

We probably will increasingly rely on AI for optimizing tasks like those and it'll contribute heavily to continued technological progress, but I don't personally see any specific turning point or runaway reaction stemming from just a self-improving AGI.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#214

Post by paxys »

keerthiko wrote: post #191
Jack000 wrote: post #116

I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…

At least we have lots of very complex simulated or pseudo-simulated environments already — throw your AGI agent into a sandbox mode game of GTA6, or like OpenAI and DeepMind already did, with DOTA2 and StarCraft II (with non-G-AIs). They have a vast almost-analog simulation space to figure out and interact with (including identifying or coming up with a goal). So while it is significant compute overhead, it at least…

I sure hope no one is planning to unleash their AGI in the real world after having it spend many (virtual) lifetimes playing GTA.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#215

Post by alexashka »

Jack000 wrote: post #116

I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…

Are you optimistic for how this AGI will get used?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#216

Post by viraptor »

semi-extrinsic wrote: post #139
Tepix wrote: post #7

I wonder if Carmacks moral compass is in order. First he sticks around at Facebook, now he endangers humanity with AGI. And i'm only half joking.

The meme that AGI, if we ever have it, will somehow endanger humanity is just stupid to me. For one, the previous US president is the perfect illustration that intelligence is neither sufficient nor necessary for gaining power in this world. And we do in fact live in a world where the upper echelons of power mostly interact in the decidedly analog spaces of leadership summits, high-end restaurants, golf courses and c…

> Furthermore, due to the warring nature of us humans, the important systems in the world like banking, electricity, industrial controls, military power etc. are either air-gapped or have a requirement for multiple humans to push physical buttons in order to actually accomplish scary things.

Well, I have bad news for you. Airgap is very rarely a thing and even when it is people do stupid things. Examples from each extreme: all the industrial control systems remote desktops you can find with shodan on one side and Stuxnet on the other.

> Sure, a mischievous AGI could do some annoying things. But nothing that our human enemies existing today couldn't also do.

Think Wargames. You don't need to do something. You just need to lie to people in power in a convincing way.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#217

Post by ge96 »

SketchySeaBeast wrote: post #160
ge96 wrote: post #86

Earlier quoted context omitted.

I wonder about this, if you had great/true automation, free energy from the sun, is there any need to do anything. As in value of money.

But who would own the automatons and power generators, and what would be their impetus to share their power? Unless the means of (energy) production moved out of the hands of the few it seems like it wouldn't make the rest of our lives any more idyllic.

Yeah it's true. When I donate/help I always feel this "mine". I believe in merit, you know, effort in effort out. It's nice to help people but there are also too many... and bad actors. So idk if it'll ever happen or just for select few anyway.

I almost regret being at this phase of life where we are aware of what's possible but we most likely not see it in our lifetime. This AGI talk, colonization of space, etc... but can strive towards it/have fun trying in the meantime.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#218

Post by Ekaros »

POiNTx wrote: post #195
TacticalCoder wrote: post #89

Does AGI implies the technological singularity and if not, why not?

It does once you have a human level AGI, it should be trivial to scale it up to a superhuman level.

I'm wondering if scaling up is trivial. Ofc, it depends on how much computational resources a working AGI needs. And if at that point they are capable of optimizing themselves further. Or optimizing production of more resources.

Still, scaling up might not be simple if we look at all the human resources currently poured in software and hardware.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#219

Post by mywittyname »

tux1968 wrote: post #175
Trasmatta wrote: post #2

Interesting to see how he's progressed with this. When he first announced he was getting into AI it sounded almost like a semi retirement thing: something that interested him that he could do for fun and solo, without the expectation that it would go anywhere. But now he seems truly serious about it. Wonder if he's started hiring yet.

He mentioned, in his Lex Friedman interview, that accepting investor money was a way to keep himself serious and motivated. He feels an obligation to those putting their money in.

Ah, I was thinking that $20MM doesn't seem like a lot of money for someone like Carmack. Surely he could have self-funded a business himself. This explains why he didn't.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#220

Post by HL33tibCe7 »

JoshTko wrote: post #138
cgrealy wrote: post #130

I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet. The idea that we can control something like that is laughable.

It's akin to nuclear weapons. If you do not develop them, then you'd be subject to the will of the ones that develops them first. So invariably you have to invest in AGI lest, an unsavory group develops it first.

Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
Post reply on HN