Earlier quoted context omitted.
What's happening?
Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug.
Not bad for 10 days of work.
John Carmack's new AGI company, Keen Technologies, has raised a $20M round
321–330 of 643 posts
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#322I believe to create AI, we need to first simulate the universe. It's the only way that makes sense to me apart from some magical algorithm people think will be discovered. I'm doubtful we'll reach it in our lifetimes, true AI running on supercomputers, it seems like the final, end all mission. Like switching your Minecraft world from survival to creative mode.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#323I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
Not sure if "optimistic" is the proper word here. Perhaps "scared senseless in the end-of-mankind kind of way" is more appropriate?
We’re all going to die.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#324I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer.
In 10 years this is going to look just as naive as when people thought AGI was imminent in the 1960s and would be based on symbolic manipulation in LISP or whatever. https://en.m.wikipedia.org/wiki/History_of_artificial_intell...
Deep learning has been great so far but we have no idea how far we are from AGI. https://www.scientificamerican.com/article/artificial-genera...
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#325Let's see if he can build an A-Team. I hope he hires Bryan Cantrill, Steve Klabnik and Chris Lattner. They are good hackers.
Bryan and Steve, who are getting https://oxide.computer/ up and running? Probably best to leave them at it! Chris has his own AI company too.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#326Earlier quoted context omitted.
Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
>> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn…
Imagine a bunch of chimps capture a human. They put them in a cage surrounded by tigers and spikes and remove all weapons from the human.
The human uses their phone to call for rescue.
That is us trying to contain an AGI. We probably cannot even conceive of the ways it can get out of any pitiful cage we put it in.
That, or it’s so dumb, it’s not worth making in the first place.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#327I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.
Which part do you find objectionable - the lack of progress in previous years or the current/future potential of transformers in RL? I do work in ML but mostly applications instead of research.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#328Earlier quoted context omitted.
How does the AGI get this magical ability to coerce people? We couldn't even get half the population to wear a face mask after bombaring them with coercion for weeks on end.
watch the film
Sorry, I just wasted ten minutes watching a YouTube video that someone else in this thread asserted was going to explain everything. Said video was entirely tangential to the point in question - why can't we just tell the AGI to sod off with it's wishes and goals. At the end of the day it is all just bits going through wires, which we are perfectly capable of disconnecting.
Now I'm not going to spend a couple of hours watching a film that has a mediocre IMDB rating. But I do notice that the synopsis says "they gave the AGI total control over the US nuclear weapons arsenal".
Yeah, let's not do that, I agree fully on this point. If we are concerned a thing will be evil and/or coercive, let's not give it weapons or other means by which to enact coercion.
This is kind of my point - the AGI won't just stumble over the nuclear codes in a Reddit thread. For the AGI to actually accomplish something of real-world relevance, us humans have to agree.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#329Earlier quoted context omitted.
It's cold if he killed it/had it euthanized. Not if he simply found it a better home where it was a better fit and more appreciated.
From Masters of Doom : Scott Miller wasn’t the only one to go before id began working on Doom. Mitzi would suffer a similar fate. Carmack’s cat had been a thorn in the side of the id employees, beginning with the days of her overflowing litter box back at the lake house. Since then she had grown more irascible, lashing out at passersby and relieving herself freely around his apartment. The final straw came when she p…
I hope he’s maybe grown up since then. Not thrilled to have that attitude from someone working on general AI.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#330Earlier quoted context omitted.
We will have something that we ourselves define to be AGI, sure, but then it's easy to hit any goal that way. Is that machine really intelligent? What does that word even mean? Can it think for itself? Is it sentient? Similar to AI, AGI is going to be a new industry buzzword that you can throw at anything and mean nothing.
> AGI Idk what prompted you to say this, but is there a version of AGI that isn't "real" AGI? I don't know how anyone could fake it. I think marketing departments might say whatever they want, but I don't see any true engineers falling for something masquerading as AGI. If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans c…
>If someone builds a machine that can unequivocally learn on it's own, replicate itself, and eventually solve ever more complex problems that humans couldn't even hope to solve, then we have AGI. Anything less than that is just a computer program.
Behold, a slime mold.