Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 144 posts

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#51

Post by someweirdperson »

mindcrime wrote: post #20
optimalsolver wrote: post #17

Carmack says he's pursuing a different path to AGI, then goes straight to the guy at the center of the most saturated area of machine learning (deep learning)? I would've hoped he'd be exploring weirder alternatives off the beaten path. I mean, neural networks might not even be necessary for AGI, but no one at OpenAI is going to tell Carmack that.

Wouldn't it be fair to say that one has to know what the current path is and have some idea where it leads and what its issues are, before forging a new path? I mean, any idiot can go off-trail and start blundering around in the weeds, and ultimately wind up tripping, falling, hitting their head on a rock, and drowning to death in a ditch. But actually finding a new, better, more efficient path probably involves at l…

To walk a path no knowledge of the existing is needed. But to be able to claim it is new it is. Even more so to be able to claim that the new is better.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#52

Post by Beldin »

RC_ITR wrote: post #42
hardware2win wrote: post #6

Earlier quoted context omitted.

I do wonder whether people behind Attention is all you need paper Will receive Turing Award It is being cited often

>Will receive Turing Award This is the weird thing - hopefully not! Hopefully there's even better NN models coming out every 5-10 years and we look back on transformers as 'just a phase' sort of like how we look back at RNN's (which were no less of an amazing achievement - look at the proliferation of voice assistants), as potentially obsolete technology today. Fore example, attention is great and does a really good…

> symbology

Off-topic, but now I have William Dafoe going "What's the 'symbology' here? The symbolism ..." in my head (from Boondock Saints).

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#54

Post by agar »

mindcrime wrote: post #20
optimalsolver wrote: post #17

Carmack says he's pursuing a different path to AGI, then goes straight to the guy at the center of the most saturated area of machine learning (deep learning)? I would've hoped he'd be exploring weirder alternatives off the beaten path. I mean, neural networks might not even be necessary for AGI, but no one at OpenAI is going to tell Carmack that.

Wouldn't it be fair to say that one has to know what the current path is and have some idea where it leads and what its issues are, before forging a new path? I mean, any idiot can go off-trail and start blundering around in the weeds, and ultimately wind up tripping, falling, hitting their head on a rock, and drowning to death in a ditch. But actually finding a new, better, more efficient path probably involves at l…

> probably involves at least some understanding of the status quo.

Oh man, you had me going with such a vivid metaphor. I was really hoping for a payoff in the end, but you abandoned it. The easy close would be "probably involves at least some understanding of the existing terrain" but I was optimistic for something less prosaic.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#57

Post by ilaksh »

My guess is that multimodal transformers will probably eventually get us most of the way there for general purpose AI.

But AGI is one of those very ambiguous terms. For many people it's either an exact digital replica of human behavior that is alive, or something like a God. I think it should also apply to general purpose AI that can do most human tasks in a strictly guided way, although not have other characteristics of humans or animals. For that I think it can be built on advanced multimodal transformer-based architectures.

For the other stuff, it's worth giving a passing glance to the fairly extensive amount of research that has been labeled AGI over the last decade or so. It's not really mainstream except maybe the last couple of years because really forward looking people tend to be marginalized including in academia.

https://agi-conf.org

Looking forward, my expectation is that things like memristors or other compute-in-memory will become very popular within say 2-5 years (obviously total speculation since there are no products yet that I know of) and they will be vastly more efficient and powerful especially for AI. And there will be algorithms for general purpose AI possibly inspired by transformers or AGI research but tailored to the new particular compute-in-memory systems.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#59

Post by mindcrime »

agar wrote: post #54
mindcrime wrote: post #20

Earlier quoted context omitted.

Wouldn't it be fair to say that one has to know what the current path is and have some idea where it leads and what its issues are, before forging a new path? I mean, any idiot can go off-trail and start blundering around in the weeds, and ultimately wind up tripping, falling, hitting their head on a rock, and drowning to death in a ditch. But actually finding a new, better, more efficient path probably involves at l…

> probably involves at least some understanding of the status quo. Oh man, you had me going with such a vivid metaphor. I was really hoping for a payoff in the end, but you abandoned it. The easy close would be "probably involves at least some understanding of the existing terrain" but I was optimistic for something less prosaic.

Sorry to disappoint. My creative juices aren't flowing today I guess. Need more coffee, or something!

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#60

Post by TimPC »

ilaksh wrote: post #57

My guess is that multimodal transformers will probably eventually get us most of the way there for general purpose AI. But AGI is one of those very ambiguous terms. For many people it's either an exact digital replica of human behavior that is alive, or something like a God. I think it should also apply to general purpose AI that can do most human tasks in a strictly guided way, although not have other characteristic…

Why do you think multimodal transformers will get us anywhere near general purpose AI? Multimodal transformers are basically a technology for sequence-to-sequence intelligent mappings and it seems to me extremely unlikely that general intelligence is one or more specific sequence-to-sequence mappings. Many specific purpose problems are sequence-to-sequence but these tend to be specialized functionalities operating in one or more specific domains.
Post reply on HN