Earlier quoted context omitted.
People don't really consider the immense risk of "speed superintelligences" as a very quick and relatively easy follow-on step to the development of AGI. Once developed, one solely needs to turn up the execution rate of an AGI, which would result in superhuman performance on most practical and economically meaningful metrics. Imagine if for every real day that passed, one experienced 100 days of subjective time. Woul…
The energy used for "training" and getting to the current human brain was huge though if you consider evolution as part of it. Billions of living beings for billion of years.
John Carmack's new AGI company, Keen Technologies, has raised a $20M round
491–500 of 643 posts
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#492Earlier quoted context omitted.
> There is an inverse relationship between the age of a futurist and the amount of time they think it will take for their predictions to become true. I think calling Carmack a Futurist is pretty insulting.
Why? Because he also wrote some game engines?
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#493Earlier quoted context omitted.
There's a hardware component here too though. I think hybrid photonic AI chips handling some of the workload are supposed to hit in 2025 at the latest, and some of the research on gains is very promising. So we may see timelines continue to accelerate as broader market shifts occur outside just software and models.
hybrid photonic AI chips handling some of the workload> working in photonics, read/reviewed papers about that and personly feel it's unlikely to happen in the next 5 years. The low density of integration, no usable nonlinearity beyond your photodetector and prohibitivly power inefficient conversion between analog(photonics) and digital make them hard to beat your Nvidia card.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#494I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#495Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#496AGI - Artificial General Intelligence (also Adjusted Gross Income)
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#497I thought AGI meant "adventure game interface". Apparently not, what a disappointment!
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#498AGI will be more dangerous that nuclear weapons. People are not allowed to start a nuclear weapon company. At all. Why are people allowed to casually start an AGI company?
This is highly debatable and frankly no one on the planet is qualified to know this for sure. It's just as likely it won't be dangerous.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#499Earlier quoted context omitted.
I agree with this. Optimists might think that the AGI won't be connected to any network, so it can't interact with the physical world. I doubt that. People will be stupid enough to have weapons controlled by that AGI (because arms race!) and then it's over. No sufficiently advanced AGI will think that humans are worth keeping around.
Once it figures out how to rewire itself to increase its intelligence, we're toast.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#500Earlier quoted context omitted.
It does once you have a human level AGI, it should be trivial to scale it up to a superhuman level.
I'm wondering if scaling up is trivial. Ofc, it depends on how much computational resources a working AGI needs. And if at that point they are capable of optimizing themselves further. Or optimizing production of more resources. Still, scaling up might not be simple if we look at all the human resources currently poured in software and hardware.
So you could pretty easily duplicate the result a couple order of magnitude times.