Live data from Hacker News

John Carmack’s ‘Different Path’ to Artificial General Intelligence

dallasinnovates.com

251–260 of 506 posts

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#251

Earlier quoted context omitted.

As soon as we reach real AI, even at the level of a 5 years old, it is game over. So education or not education people will become completely useless to the rich that will own the AI

Useless? They still have to buy all that crap that is currently sold to them.

No need to sell things at that point. Just produce and consume whatever you want.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#252
post #197

What would we actually consider an AGI? Wikipedia lists the following tests > The Turing Test (Turing): A machine and a human both converse unseen with a second human, who must evaluate which of the two is the machine, which passes the test if it can fool the evaluator a significant fraction of the time. Note: Turing does not prescribe what should qualify as intelligence, only that knowing that it is a machine should…

I think GTP is very far from solving 1), at least if you happen to know anything about LLMs. I think the easiest one of these would be 4) actually.

Is it? I remember reading [1] back in the day, which showed some limitations of GPT3, but ChatGPT actually answers all of those perfectly.

[1] https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.h...

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#253

Earlier quoted context omitted.

Wouldn’t this be literal slavery? AGI = a person Instantiating people for work and ending their existence afterward seems like the virtual hell that Iain M Banks and Harlan Ellison wrote about. https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...

There is a horror story written about this theme, went viral a few years back https://qntm.org/mmacevedo Please do give it a quick read.

It's hardly a story.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#254
post #241
post #213

Earlier quoted context omitted.

People can learn skills from books, which are entirely passive. The learning process ultimately resides within the student; issues of motivation, morale, direction, diligence, discipline, time, and mental health matter a lot more than just going through some material.

Try to learn Jiu-Jitsu from a book and then go into an actual fight to see how well it works.

Guess we don't have to worry about AI taking that job, then.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#255

Earlier quoted context omitted.

It can do some trivial problems, but not others, and it can't consistently do real math (proving theorems), because it doesn't understand logic. It can only imitate it some of the time for trivial or often-presented proofs when the necessary steps have become embedded in its next-token-generation model. It's still getting this wrong, for instance: Prompt: You call heads/tails for 1000 coin tosses. You know the coin b…

Seems to have improved already. Just tried the coin toss prompt and get the following Your strategy should be to always call "heads," since it has a higher probability of landing on heads (52%) compared to tails (48%). By consistently choosing heads, you'll earn more money in the long run. And to prove taht the square root of 2 is irrational: The proof that the square root of 2 is irrational is a well-known proof tha…

This "proof" is actually a pretty good manifestation of the current limitations of GPT3; it mangles together two different proofs in a way that does not make sense.

It starts by setting up for an elementary proof (a/b = sqrt(2) with a, b minimal implies 2|a^2 implies 2|a implies 4|a^2 implies 2|b^2 implies 2|b which is a contradiction). But instead of following through it finishes using another proof based on the rational root theorem, and makes up some bullshit on the way to make the argument flow nicely (it does not follow from a^2 / b^2 = 2 with no common factors that 2 is a perfect square).

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#256
post #192

Earlier quoted context omitted.

Wouldn’t this be literal slavery? AGI = a person Instantiating people for work and ending their existence afterward seems like the virtual hell that Iain M Banks and Harlan Ellison wrote about. https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...

Would turning one off be murder? Or does that only apply to deletion?

There will probably be a gig economy, where you can pay spot rates for an idle Frank that could get a page and need to leave at any time.

Or maybe they'll handle things like call centers and 911 dispatch in their spare time.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#257

> So, about four years ago, I went on one of my week-long retreats, where I just take a computer and a stack of reference materials and I spend a week kind of reimplementing the fundamentals of the industry. And getting to the point where it’s like, ‘All right, I understand this well enough to have a serious conversation with a researcher about it.’ And I was pretty excited about getting to that level of understandin…

Anything you study for a few months you can become the world's leading expert on. It's a lesson I learned while doing my Ph. D. That's all it takes. After a few weeks, you get to the point where there are only a few others in the world that have read and are able to understand what you've read. A few months on, you are generating new ideas and insights. They might be wrong. But they won't be uninformed. John Carmack…

If it is simply about linear algebra, can you please read this ML Paper[1], go through all the proofs and lemmas over a week? You already have a PhD in ML, should be easy. Every kid graduating in STEM understands/should understand linear algebra. Knowing linear algebra is such a low bar.

[1] https://arxiv.org/pdf/1904.09237

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#258
post #254
post #241

Earlier quoted context omitted.

Try to learn Jiu-Jitsu from a book and then go into an actual fight to see how well it works.

Guess we don't have to worry about AI taking that job, then.

Maybe we'll see some sorts of manual labor as the last bastion of not automated, human performed work. Of the kind that demands a lot both from the human motor skills and also higher thinking processes.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#259

Carmack is clearly a brilliant guy, but it feels like he's fallen into the trap of overfitting on his previous successes and believing they generalise into other domains. No doubt his experience and innovations in computer graphics gives him a good insight into problems in vision, etc. but I don't see anything particularly original or orthogonal in what he's saying with regards to "general AI".

What makes this a case of overfitting success vs. one of applying a lifetime of experience?

I’m not claiming the first can’t exist, but I see no reason to conclude that is the case here.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#260
post #100
post #22

Would be interesting to get a list of those 40 papers mentioned

Maybe ask him on Twitter for it?

They asked on Twitter and he didn’t reply. We need someone with a blue check mark to ask. https://twitter.com/ifree0/status/1620855608839897094
Post reply on HN