Earlier quoted context omitted.
Counter-argument: DALL-E is smart enough to understand that an astronaut riding a horse makes more sense than a horse riding an astronaut, and therefore assumes that you meant "a horse-riding astronaut" unless you go out of your way to specify that you definitely do, in fact, want to see a horse riding an astronaut.
Because intelligence is more than frequentism. Being able to predict that a dice lands on a given side with probability 1/6 is not a demonstration of intelligence. It feels a bit naive to even suggest this and I suspect that you aren't a ML researcher (and if I'm right, maybe don't have as much arrogance because you don't have as much domain knowledge).
John Carmack's new AGI company, Keen Technologies, has raised a $20M round
621–630 of 643 posts
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#622Earlier quoted context omitted.
I don't see it. Did they mean "uncanny valley"?
Kinda. The gullibility gap more refers to the fact that it is really easy to believe the machines are smarter than they are because they can perform tasks that nothing but humans can do. With the exception of maybe parrots there's no other creatures that can talk so we associate this with intelligence. But I think it is rather naive to believe that the ability to string together reasonable sounding text is actually i…
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#623Earlier quoted context omitted.
I know it maybe too much to ask but as a noob to this field I’d love to hear more about why AGI is not feasible even in 50 years.
Sure. It is rather complex to be honest but I'll do my best to give a high level abstract. I should first off explain that there's no clear definition of intelligence. You might see some of my other comments use the term HLI, which is Human Level Intelligence. It is a term gaining in popularity due to this difficult to define definition and allows for more a porn like definition -- I know it when I see it. Philosophe…
> Philosophers, psychologists, neuroscientists, and many others have been contemplating the definition of intelligence for centuries and we're still not -quite there
Given the decade-old, seminal, widely cited papers by Legg (co-founder of Deepmind) & Hutter (long-term professor of ANU with multiple awards, currently researcher at Deepmind) titled "Universal Intelligence: A Definition of Machine Intelligence"[1] and "A Collection of Definitions of Intelligence"[2] and the whole rigorous mathematical theory of universal bayesian reinforcement learning aka "AIXI"[3] developed during the last decades which you fail to mention, this soft language of "not quite there" looks evasive, bordering on disingenious even.
Citing Chomsky, with his theorems which were interesting at their own time yet quickly faded into irrelevance once the modern assumptions of learning theory were developed, and Marcus whose simplistic prompts are a laughing stock among degreed and amateur practicing researchers alike brings little of value to the already biased discussion.
> Of course, there are others in the field that argue that the algorithms can learn this through enough data
Such as Rich Sutton, one of the fathers of Reinforcement Learning as an academic discipline, whose recent essay "Bitter Lesson"[4] took the field by word of mouth, if nothing else. It takes a certain amount of honesty and resolution to acknowledge that one's competitive pursuit of knowledge, status and fame via participating in academic "publish or perish" culture does not bring humanity forward along a chosen axis, compared to more mundane engineering-centric research, many people commend him for that.
> So another thing, is that machines haven't shown really good generalization, abstraction, or really inference.
To play a devil's advocate: "Scaling is all you need" and "Attention is all you need" are devilishly simple hypotheses which seem to work again and again when we reluctantly dole out more compute to push them further. With a combination of BIG-bench[5] and an assortment of RL environments as the latest measuring stick, why should we, as a society, avoid doling out enough compute to take it by storm in favor of a certain hypothesis implicit to your writing, which could as well be named "Diverse academic AI funding is all you need"? Especially when variations of this hypothesis have failed us for decades starting with the later part of the XX century, at least if we consider AI research.
> but let's recognize that this camp does exist (we have no facts, so there's opinions).
With all due respect, the continuing lavish (by global standards) funding of your career and careers of many other unlucky researchers should not be a priority when such massive boon to all of humanity as a practical near-HLI/AGI is at stake. "Think of the children", "think of the ill", think of everybody who is not yourself - I say as a taxpayer, hoping for AGI checkpoint to be produced by academics and not the megacorporations, and owned by the people, and not the select few.
Which - the proprietary nature of FAANG-developed AI - might be a real problem we are going to face this decade which is going to overshadow any career anxiety you may have experienced so far.
1. https://arxiv.org/abs/0712.3329
2. https://arxiv.org/abs/0706.3639
3. https://arxiv.org/abs/cs/0004001
4. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#624Earlier quoted context omitted.
Buy compute with stolen cryptocurrency. Convince a couple rich people that it can cure aging with _just_ enough compute. Hack datacenters (an AGI would likely be much much much better at finding security holes than humans). Make _really really funny videos_ and create the most popular Patreon of all time. If you're 500 IQ or above there are a _lot_ of things you can do, and probably simultaneously, since an AGI proba…
Sure, this will help with some initial growth in computing resources, if this is seen as a required goal by an "evil" AGI agent before the destruction of humans. And it will have an easy time to hide in the growing computing industry. But if it e.g. kills all humans, it's suddenly left with a limited number of chips and can't grow beyond that number, without being able to manufacture new chips. It can't survive for r…
I imagine an better-than-human AGI could figure this out and plan/act accordingly, considering a mere human being figured this out. (Also sometimes it might be better to think of an AGI, when it gets to a certain level, as a nation state and not an individual)
> trivial
It doesn't need to be trivial to be incredibly dangerous
> The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#625Earlier quoted context omitted.
>Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious. >I feel like any AGI developed in silico without freedom of movement wil…
> It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious. Is it? Is there proof that consciousness is not a requirement to close-to-human level intelligence? Given that we do not even know how consciousness works in our brains, I don't think we have the answer yet. I'm not sure it's plausible to make such an assumption.
Possibly a fair argument, but I think the opposite would be just as valid then. That you can't assume that consciousness IS a requirement for intelligence.
It seems hard to argue that you don't need some intelligence for consciousness, there seems to be a kind of minimum. We can already see that narrow intelligences for Vision, NLP, game playing, etc are at or above human level and no one claims these systems are conscious. So why should we assume that some meta system of these already existing systems would be?
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#626Earlier quoted context omitted.
> we didn't have the theory or the models, there was no path forward and so it was impossible to predict. Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.
> As someone working in AI research Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence. I think people (especially tech enthusiasts) are getting caught in the gullibility gap.
Look at the work result of folks like engineers at Google - state of the art in AI and driving most of the trends in AGI. Academic opinions are shot down quite quickly there, PhD hiring is mostly frowned on, even in teams working on AGI related tech.
It might be expected that PhD experience provides surprising little insight here unfortunately. Academic smugness is not something that should be offered as part of an opinion- just like many computer science related concepts, progress moves so quickly in these topics that by the time academics reach and start to iterate on concepts, companies have already moved very far beyond.
This isn't intended to be condescending to academics, just real life experience I've had working closely with both crowds and seeing this firsthand. It's a very palpable thing in industry, an Academic gives an opinion on AGI and engineers roll their eyes - since Academic perspectives are currently so useless or out of date for any current, meaningful progress.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#627Earlier quoted context omitted.
Yeah he didn't elaborate on this in the interview, which I would have liked. I should amend my comment to say that "6 or so breakthroughs" sounds a lot more plausible to me than "1 breakthrough" or "just scaling", which you will see some people advocate, including in this thread. That's what I meant. I believe the latter is fundamentally mistaken and those people simply don't understand what they don't understand (in…
What do you mean exactly when you say “deep learning”?
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#628Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#629Earlier quoted context omitted.
Kind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
What makes humans special though? AI will be born on Earth so we will share that common trait with it, but perhaps one should have humility to accept that future doesn’t belong to a weak species like humans.
We’re the most intelligent thing we’ve discovered in the universe. Also, I am human, so I have a vested interest in bad stuff not happening to humans.
> AI will be born on Earth so we will share that common trait with it
Covid-19 was also born on earth. That wasn’t too flash for humanity.
> perhaps one should have humility to accept that future doesn’t belong to a weak species like humans.
Not sure how you class humans as a “weak species”? Unless you’re comparing organically life forms to hypothetical digital life?
Anyway, I suspect if any kind of life form could expand to the stars, it’d be digital.
Re: John Carmack's new AGI company, Keen Technologies, has raised a $20M round
#630Earlier quoted context omitted.
I'm 100% serious. I literally don't understand your concern at all. >>And as soon as it has access to the internet, it can copy itself. So can viruses, including ones that can "intelligently" modify themselves to avoid detection, and yet this isn't a major problem. How is this any differenent? >>Good luck pulling the plug of the internet. I could reach down and pull my ethernet cable out but it would make posting thi…
> So can viruses, including ones that can "intelligently" modify themselves to avoid detection, and yet this isn't a major problem. How is this any different? I now regret spending half an hour writing a response to your earlier comment. You can't tell why a mass deployment of motivated autonomous "NSO-9000s" might be more dangerous than a virus that just changes its signature to fool a executable scanner? I don't be…
>>You can't tell why a mass deployment of motivated autonomous "NSO-9000s" might be more dangerous than a virus that just changes its signature to fool a executable scanner? I don't believe you.
So I just want to address that - of course I can tell the difference, but I just can't believe we will arrive at that level of crazy intelligence straight away. Like others have said - more like AGI with the capacity to learn like a small child, then years of training later they grow to be something like an adult. More human facsimile less HAL9000.
I only brought up self modifying viruses because that's the example of current tech that's extremely incentivised to multiply and avoid detection, that's its main reason for existence, and it does that very poorly.