Earlier quoted context omitted.
> Who is we exactly? When I read these kind of threads, I believe it's "enthusiast" laypeople who follow the headlines but don't actually have a deep understanding of the tech. Of course there are the promoters who are raising money and need to frame each advance in the most optimistic light. I don't see anything wrong with that, it just means that there will be a group of techie but not research literate folks who a…
Can someone please explain like we are fifteen why AGI is impossible, at least right now? Or if not AGI, then something similar to a cat/etc mind? As far as I am imagining it, current models are pipelines of various trained networks (and more traditional filters in the mix) that operate like request-reply. Why can’t you just connect few different pipelines in a loop/graph and make an autonomous self-feeding entity? B…
John Carmack's new AGI company, Keen Technologies, has raised a $20M round
401–410 of 643 posts
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#402Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentall…
>Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious. >I feel like any AGI developed in silico without freedom of movement wil…
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#403Earlier quoted context omitted.
> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.
> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#404Earlier quoted context omitted.
>> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn…
On the off chance that you're serious: Even if you can pull the plug before it is too late, less moral people like Meta Mark will not unplug theirs. And as soon as it has access to the internet, it can copy itself. Good luck pulling the plug of the internet.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#405Earlier quoted context omitted.
Which part do you find objectionable - the lack of progress in previous years or the current/future potential of transformers in RL? I do work in ML but mostly applications instead of research.
I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL. I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.
10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#406Earlier quoted context omitted.
watch the film
Sorry, I just wasted ten minutes watching a YouTube video that someone else in this thread asserted was going to explain everything. Said video was entirely tangential to the point in question - why can't we just tell the AGI to sod off with it's wishes and goals. At the end of the day it is all just bits going through wires, which we are perfectly capable of disconnecting. Now I'm not going to spend a couple of hour…
your loss
> For the AGI to actually accomplish something of real-world relevance, us humans have to agree.
Donald Trump managed to get control of the US nuclear arsenal
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#407Earlier quoted context omitted.
I was surprised by how bullish he is about this. At least a few years ago the experts in the field didn't see AGI anywhere near us for at least a few decades, and all of the bulls were physicists, philosophers or Deepak-Chopra-for-the-TED-crowd bullshit artists who have never written a line of code in their lives, mostly milking that conference and podcast dollar, preaching Skynet-flavored apocalypse or rapture. To s…
if Carmack's in I'm in. Has he ever been drastically wrong?
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#408Earlier quoted context omitted.
From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.
Thank you for this comment. I'd never really considered this and it is blowing my mind.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#409Earlier quoted context omitted.
> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.
I know it maybe too much to ask but as a noob to this field I’d love to hear more about why AGI is not feasible even in 50 years.
Philosophers, psychologists, neuroscientists, and many others have been contemplating the definition of intelligence for centuries and we're still not quite there. But we do have some good ideas. One of the important aspects is the ability to draw causal relationships. This is something you'll find state of the art (SOTA) text to image (T2I) generators fail at. The problem is that these models learn through a frequency based relationship with the data. While we can often get the desired outcome through significant prompt engineering it still demonstrates a lack of the algorithm really understanding language. The famous example is "horse riding an astronaut." Without prompt engineering the T2I models will always produce an astronaut riding a horse. This is likely due to the frequency of that relationship (people are always on horses and not the other way around).
Of course, there are others in the field that argue that the algorithms can learn this through enough data, model scale, and the right architecture. But this also isn't how humans learn and so we do know that if this does create intelligence it might look very different ("If a lion could talk I wouldn't be able to understand it"). Obviously I'm in the other camp, but let's recognize that this camp does exist (we have no facts, so there's opinions). I think Chomsky makes a good point: our large language models have not taught us anything meaningful about language. Gary Marcus might be another person to look into as he writes a lot on his blogs and there's a podcast I like called Machine Learning Street Talk (they interview both Marcus and Chomsky but also interview people that have the opposite opinion).
So another thing, is that machines haven't shown really good generalization, abstraction, or really inference. You can tell a human a pretty fictitious scenario and we're really good at producing those images. Think of all the fake scenarios you go through while in the shower or waiting to fall asleep. This is an important aspect of human intelligence as it helps train us. But also, we have a lot of instincts and a lot of knowledge embedded in our genetics (as does every other animal). So we're learning a lot more than what we can view within our limited lifetimes (of course our intelligence also has problems). Maybe a good example of abstraction is "what is a chair." There's a whole discussion about embodiment (having a body) and how language like this may not be able to be captured in the abstract manner without a body since a chair is essentially anything that you can sit on.
I feel like I've ranted and may have written a mess of a lot of different things haha. But I hope this helps. I'll try to answer more and I'm sure others will lay out their opinions. Hopefully we can see a good breadth of opinions and discussion here. Honestly, no one knows the answers.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#410Also, several Keen Technologies domains already exists in various forms. They're probably going to get a lot of traffic today.