Live data from Hacker News

John Carmack's new AGI company, Keen Technologies, has raised a $20M round

twitter.com

401–410 of 643 posts

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#401
post #380

Earlier quoted context omitted.

> Who is we exactly? When I read these kind of threads, I believe it's "enthusiast" laypeople who follow the headlines but don't actually have a deep understanding of the tech. Of course there are the promoters who are raising money and need to frame each advance in the most optimistic light. I don't see anything wrong with that, it just means that there will be a group of techie but not research literate folks who a…

Can someone please explain like we are fifteen why AGI is impossible, at least right now? Or if not AGI, then something similar to a cat/etc mind? As far as I am imagining it, current models are pipelines of various trained networks (and more traditional filters in the mix) that operate like request-reply. Why can’t you just connect few different pipelines in a loop/graph and make an autonomous self-feeding entity? B…

Cool, make it happen. Don’t know how? Neither does anyone else. That’s why it’s not happening right now.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#402

Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentall…

>Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious. >I feel like any AGI developed in silico without freedom of movement wil…

Good point. The lines between consciousness and knowledge seem blurred too since though we may have certain kinds of knowledge without embodiment (such as 2+2=4), other knowledge such as qualia [1] may be inaccessible to an unembodied agent. The classic example of "Mary knowing everything about the color red before having seen the color red physically. Does she still learn more about the color red when she actually experiences it with her own eyes?"

[1] https://plato.stanford.edu/entries/qualia/

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#403

Earlier quoted context omitted.

> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.

> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.

As an AI researcher myself I can definitely see why people used to have these sentiments, but I also have to point out how the big papers of the last ~12 months or so have changed the landscape. Especially multi-modal models like Gato. To my surprise, a lot of people (even inside the community) still tend to put human intelligence on some sort of pedestal, but I believe once a system (or host of systems) achieves a sufficient level of multi-modality, these same people will no longer be able to tell human and machine intelligence apart - and in the end that is the only thing that counts, since we lack a rigorous definition of "understanding" or consciousness as a whole. Transformers are certainly going to become even more general purpose in the future and I'm sure we haven't even seen close to what's possible with RL. Regarding the AGI timescale, I'm a bit more bullish. Not Elon-type "possible end-all singularity in 5 years" but more Carmack-like "learning disabled toddler running on 10k+ GPUs by the end of the decade." Whether RL+transformers will be able to go all the way there is impossible to tell right now, but if I was starting an AGI company today that's definitely the path I would pursue.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#404
post #274

Earlier quoted context omitted.

>> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn…

On the off chance that you're serious: Even if you can pull the plug before it is too late, less moral people like Meta Mark will not unplug theirs. And as soon as it has access to the internet, it can copy itself. Good luck pulling the plug of the internet.

...and even if Mark's bot is dogshit, Musk will cheerfully sell embodied versions by the millions to replace workers, systematically handing over our industrial infrastructure first to an unrestrained capital class and then to the robots. I'm not even sure which will be worse.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#405

Earlier quoted context omitted.

Which part do you find objectionable - the lack of progress in previous years or the current/future potential of transformers in RL? I do work in ML but mostly applications instead of research.

I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL. I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.

I think it's fair to say that no one knows if AGI is even possible at this point, but RL+transformer is definitely the most promising approach imo.

10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#406

Earlier quoted context omitted.

watch the film

Sorry, I just wasted ten minutes watching a YouTube video that someone else in this thread asserted was going to explain everything. Said video was entirely tangential to the point in question - why can't we just tell the AGI to sod off with it's wishes and goals. At the end of the day it is all just bits going through wires, which we are perfectly capable of disconnecting. Now I'm not going to spend a couple of hour…

> Now I'm not going to spend a couple of hours watching a film that has a mediocre IMDB rating.

your loss

> For the AGI to actually accomplish something of real-world relevance, us humans have to agree.

Donald Trump managed to get control of the US nuclear arsenal

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#407

Earlier quoted context omitted.

I was surprised by how bullish he is about this. At least a few years ago the experts in the field didn't see AGI anywhere near us for at least a few decades, and all of the bulls were physicists, philosophers or Deepak-Chopra-for-the-TED-crowd bullshit artists who have never written a line of code in their lives, mostly milking that conference and podcast dollar, preaching Skynet-flavored apocalypse or rapture. To s…

if Carmack's in I'm in. Has he ever been drastically wrong?

What about his bet on MegaTextures?

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#408

Earlier quoted context omitted.

From the point when an AGI is capable of constructing a slightly better version of itself and has the urge to do so, everything can happen very fast.

Thank you for this comment. I'd never really considered this and it is blowing my mind.

This is what some people call "the singularity" - once an intelligence can upgrade itself it can design better machines than we can, which can design better machines than they can... technology will leap ahead at a rate we won't be able to comprehend.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#409

Earlier quoted context omitted.

> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.

I know it maybe too much to ask but as a noob to this field I’d love to hear more about why AGI is not feasible even in 50 years.

Sure. It is rather complex to be honest but I'll do my best to give a high level abstract. I should first off explain that there's no clear definition of intelligence. You might see some of my other comments use the term HLI, which is Human Level Intelligence. It is a term gaining in popularity due to this difficult to define definition and allows for more a porn like definition -- I know it when I see it.

Philosophers, psychologists, neuroscientists, and many others have been contemplating the definition of intelligence for centuries and we're still not quite there. But we do have some good ideas. One of the important aspects is the ability to draw causal relationships. This is something you'll find state of the art (SOTA) text to image (T2I) generators fail at. The problem is that these models learn through a frequency based relationship with the data. While we can often get the desired outcome through significant prompt engineering it still demonstrates a lack of the algorithm really understanding language. The famous example is "horse riding an astronaut." Without prompt engineering the T2I models will always produce an astronaut riding a horse. This is likely due to the frequency of that relationship (people are always on horses and not the other way around).

Of course, there are others in the field that argue that the algorithms can learn this through enough data, model scale, and the right architecture. But this also isn't how humans learn and so we do know that if this does create intelligence it might look very different ("If a lion could talk I wouldn't be able to understand it"). Obviously I'm in the other camp, but let's recognize that this camp does exist (we have no facts, so there's opinions). I think Chomsky makes a good point: our large language models have not taught us anything meaningful about language. Gary Marcus might be another person to look into as he writes a lot on his blogs and there's a podcast I like called Machine Learning Street Talk (they interview both Marcus and Chomsky but also interview people that have the opposite opinion).

So another thing, is that machines haven't shown really good generalization, abstraction, or really inference. You can tell a human a pretty fictitious scenario and we're really good at producing those images. Think of all the fake scenarios you go through while in the shower or waiting to fall asleep. This is an important aspect of human intelligence as it helps train us. But also, we have a lot of instincts and a lot of knowledge embedded in our genetics (as does every other animal). So we're learning a lot more than what we can view within our limited lifetimes (of course our intelligence also has problems). Maybe a good example of abstraction is "what is a chair." There's a whole discussion about embodiment (having a body) and how language like this may not be able to be captured in the abstract manner without a body since a chair is essentially anything that you can sit on.

I feel like I've ranted and may have written a mess of a lot of different things haha. But I hope this helps. I'll try to answer more and I'm sure others will lay out their opinions. Hopefully we can see a good breadth of opinions and discussion here. Honestly, no one knows the answers.

Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round

#410
He doesn't seem to have purchased keen.ai yet, though it seems like it's still for sale. I just naturally went there to see the company info and saw the landing page saying it was available. If they want it, they better move quick. I see an arbitrage opportunity...

Also, several Keen Technologies domains already exists in various forms. They're probably going to get a lot of traffic today.

Post reply on HN