281–290 of 643 posts
Earlier quoted context omitted.
none of these things are air-gapped once you have the ability to coerce people if you want a fictional example: watch Colossus: The Forbin Project
How does the AGI get this magical ability to coerce people? We couldn't even get half the population to wear a face mask after bombaring them with coercion for weeks on end.
Earlier quoted context omitted.
> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. People have thought Deep RL would lead to AGI since practically the beginning of the deep learning revolution, and likely significantly before. It's the most intuitive approach by a longshot (even depicted in movies as agents receiving positive/negative reinforcement from their environment), but t…
They have? That's the approach they are using? Because that doesn't mesh well with practical reality. Where AGI use Deep RL, it's to improve on vision tasks like object classification, none of them are making any driving decisions - that seems to remain the domain of I guess you could call it discrete logic .
https://ieeexplore.ieee.org/document/9351818
B. R. Kiran et al., "Deep Reinforcement Learning for Autonomous Driving: A Survey," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 4909-4926, June 2022, doi: 10.1109/TITS.2021.3054625.
I haven't read the paper, so this is not a reading recommendation. Just posting as evidence that there is work in the area.
Earlier quoted context omitted.
There is a story in “Masters of Doom” about Carmack getting rid of his cat because “she was having a net negative effect on my life”. That’s cold.
It's cold if he killed it/had it euthanized. Not if he simply found it a better home where it was a better fit and more appreciated.
Scott Miller wasn’t the only one to go before id began working on Doom. Mitzi would suffer a similar fate. Carmack’s cat had been a thorn in the side of the id employees, beginning with the days of her overflowing litter box back at the lake house. Since then she had grown more irascible, lashing out at passersby and relieving herself freely around his apartment. The final straw came when she peed all over a brand-new leather couch that Carmack had bought with the Wolfenstein cash. Carmack broke the news to the guys.
“Mitzi was having a net negative impact on my life,” he said. “I took her to the animal shelter. Mmm.”
“What?” Romero asked. The cat had become such a sidekick of Carmack’s that the guys had even listed her on the company directory as his significant other–and now she was just gone? “You know what this means?” Romero said. “They’re going to put her to sleep! No one’s going to want to claim her. She’s going down! Down to Chinatown!”
Carmack shrugged it off and returned to work. The same rule applied to a cat, a computer program, or, for that matter, a person. When something becomes a problem, let it go or, if necessary, have it surgically removed.
I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
My evidence? OpenWorm [1]. OpenWorm is an effort to model the behaviour of a worm that has 302 mapped neurons. 302. Efforts so far have fallen way short of the mark.
How many neurons does a human brain have? 86 billion (according to Google).
I've seen other estimates that the computational power of the brain is roughly estimated as 10^15 operations per second. I suspect that's on the low end. We can't even really get that level of computation in one place for practical reasons (ie interconnects).
Neural structure changes. The neurons themselves change internally.
I still think AGI is very far off.
I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. People have thought Deep RL would lead to AGI since practically the beginning of the deep learning revolution, and likely significantly before. It's the most intuitive approach by a longshot (even depicted in movies as agents receiving positive/negative reinforcement from their environment), but t…
It may not be immediately clear, but it is nevertheless unfortunately clear from RL papers which provide adequate sample-size or compute ranges that RL appears to follow scaling laws (just like everywhere else anyone bothers to test). Yeah, they just get better the same way that regular ol' self-supervised or supervised Transformers do. Sorry if you were counting on 'RL doesn't work' for safety or anything.
If you don't believe the basic existence proofs of things like OA5 or AlphaStar, which work only because things like larger batch sizes or more diverse agent populations magically make notoriously-unreliable archs work, you can look at Jones's beautiful AlphaZero scaling laws (plural) work https://arxiv.org/abs/2104.03113 , or browse through relevant papers https://www.reddit.com/r/mlscaling/search?q=flair%3ARL&restr... https://www.gwern.net/notes/Scaling#ziegler-et-al-2019-paper Or GPT-f. Then you have stuff like Gato continuing to show scaling even in the Decision Transformer framework. Or consider instances of plugging pretrained models into RL agents, like SayCan-PaLM most recently.
I'm very optimistic for near-term AGI (10 years or less). Even just a few years ago most in the field would have said that it's an "unknown unknown", we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. The issue is that unlike supervised training y…
[1] https://venturebeat.com/business/yann-lecuns-vision-for-crea...
I thought AGI meant "adventure game interface". Apparently not, what a disappointment!
Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentall…
Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious.
>I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.
An AGI will be defacto incomprehensible to Humans. Being developed in Silicon will have little bearing on that fact.
Earlier quoted context omitted.
I was surprised by how bullish he is about this. At least a few years ago the experts in the field didn't see AGI anywhere near us for at least a few decades, and all of the bulls were physicists, philosophers or Deepak-Chopra-for-the-TED-crowd bullshit artists who have never written a line of code in their lives, mostly milking that conference and podcast dollar, preaching Skynet-flavored apocalypse or rapture. To s…
if Carmack's in I'm in. Has he ever been drastically wrong?