Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 144 posts

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#111

Post by sho_hn »

bathtub365 wrote: post #107
klabb3 wrote: post #94

Earlier quoted context omitted.

I think there’s broad consensus that slavery only applies to human labor. Even within that spectrum people avoid the term (see forced prison labor). We also don’t use it for animal labor, for instance.

Human slaves were often considered to be less than human or, at the very least, not deserving of basic rights that other humans enjoyed, as part of the moral and ethical frameworks that supported the practice. I think we might see the same shift in dominant ideology if we do have “true” AGI. I’m sure I could be convinced that an intelligence that develops and grows over a number of years begins to have a right to exi…

Given the outcry/backlash over Dall-E/ChatGPT (what is "real art", etc.) and how much of our society is permeated by a search for authenticity (perceived) already, I wonder if you're right. We might decide "artifical" lifeforms are a lower class than "evolved in nature". For many religions this could be a natural take - made by God vs. folly of man, etc.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#113

Post by ilaksh »

TimPC wrote: post #60
ilaksh wrote: post #57

My guess is that multimodal transformers will probably eventually get us most of the way there for general purpose AI. But AGI is one of those very ambiguous terms. For many people it's either an exact digital replica of human behavior that is alive, or something like a God. I think it should also apply to general purpose AI that can do most human tasks in a strictly guided way, although not have other characteristic…

Why do you think multimodal transformers will get us anywhere near general purpose AI? Multimodal transformers are basically a technology for sequence-to-sequence intelligent mappings and it seems to me extremely unlikely that general intelligence is one or more specific sequence-to-sequence mappings. Many specific purpose problems are sequence-to-sequence but these tend to be specialized functionalities operating in…

A couple of things.

1) As I said, many people have different ideas of what we are talking about. I assume that for you general purpose AI has more capabilities, such as the ability to quickly learn tasks to a high level on the fly. For me, it still qualifies as general purpose if it can do most tasks but relies on a lot of pre-training and let's say knowledgebase look up.

2) It seems obvious to me that ChatGPT proves a general purpose utility for these types of LLMs, and it is easy to speculate that something similar but with visual input/output also will be even more general. And so we are just looking at a matter of degree by that definition.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#114

Post by abrichr »

sillysaurusx wrote: post #104
abrichr wrote: post #95

Earlier quoted context omitted.

Theano was first released in 2007.

That’s actually fascinating. Were there many experiments done in it back in the 00’s? I’m just trying to imagine the things you could do with it back then. 2007 had relatively fast gpus for the time, but certainly nothing compared to today. Yet it’d certainly be enough for MNIST training, which makes me wonder what else could be done.

You can look at Yoshua Bengio's Google Scholar profile [1] and scroll down to see what they were working on around that time.

Here are some papers with many citations:

- An empirical evaluation of deep architectures on problems with many factors of variation [2]

- Extracting and composing robust features with denoising autoencoders [3]

- Scaling learning algorithms towards AI [4]

[1] https://scholar.google.com/citations?hl=en&user=kukA0LcAAAAJ...

[2] https://scholar.google.com/citations?view_op=view_citation&h...

[3] https://scholar.google.com/citations?view_op=view_citation&h...

[4] https://scholar.google.com/citations?view_op=view_citation&h...

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#115

Post by mike_d »

klabb3 wrote: post #94
sho_hn wrote: post #90

> "You’ll find people who can wax rhapsodic about the singularity and how everything is going to change with AGI. But if I just look at it and say, if 10 years from now, we have ‘universal remote employees’ that are artificial general intelligences, run on clouds, and people can just dial up and say, ‘I want five Franks today and 10 Amys, and we’re going to deploy them on these jobs,’ and you could just spin up like…

I think there’s broad consensus that slavery only applies to human labor. Even within that spectrum people avoid the term (see forced prison labor). We also don’t use it for animal labor, for instance.

If we ever conjure a way to capture the human consciousness and preserve it before death, "AI" will be based on indentured servitude.

The people given a second chance at life will be the ones who are quickest at identifying traffic signals or fire hydrants from a line up of images.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#116

Post by icepat »

unixhero wrote: post #33
icepat wrote: post #31

Earlier quoted context omitted.

Having an animal killed for no good reason, other than it caused you problems, is a bit twisted. In situations like this, rehoming them is the ethical thing to do. This is literally saying 'I don´t care if he´s killing kittens'.

Ask any vet if this happens all the time.

Yeah, but that does not mean it's not wildly unethical.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#117

Post by robotburrito »

sho_hn wrote: post #90

> "You’ll find people who can wax rhapsodic about the singularity and how everything is going to change with AGI. But if I just look at it and say, if 10 years from now, we have ‘universal remote employees’ that are artificial general intelligences, run on clouds, and people can just dial up and say, ‘I want five Franks today and 10 Amys, and we’re going to deploy them on these jobs,’ and you could just spin up like…

I wonder if those Franks and Amys would just think they are working remote jobs hammering out tickets from their studio apartments lol.

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#118

Post by 93po »

ramraj07 wrote: post #87
optimalsolver wrote: post #17

Carmack says he's pursuing a different path to AGI, then goes straight to the guy at the center of the most saturated area of machine learning (deep learning)? I would've hoped he'd be exploring weirder alternatives off the beaten path. I mean, neural networks might not even be necessary for AGI, but no one at OpenAI is going to tell Carmack that.

This is pretty much the same deal in biology as well. At calico, at verily, at CZI, even at Allen, same story - they say they will reinvent biology research and then go get the same narrow minded professors and CEOs who run the status quo and end up as one more of the same stuff. Neuralink is the only place where this pattern seemed to break a bit but then seems like Elon came into his own path with trying to push fo…

> breaking basic ethics

This didn't happen

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#119

Post by sinenomine »

optimalsolver wrote: post #17

Carmack says he's pursuing a different path to AGI, then goes straight to the guy at the center of the most saturated area of machine learning (deep learning)? I would've hoped he'd be exploring weirder alternatives off the beaten path. I mean, neural networks might not even be necessary for AGI, but no one at OpenAI is going to tell Carmack that.

The amount of disdain academically inclined people express towards reductionist engineering-first paradigms is hilarious and depressing.

The denial of obviously fertile paradigm feels like such a useless self-defeating loss to indulge in an intellectual status game.

We could be all better off right now if connectionists were given DOE-grade supercomputers in the 90s, and were supplied with custom TPUs later in the 00s as their ideas were proven generally correct via rigorous experimentation on said DOE supercomputers. This didn't happen due to what amounts to academic bullying culture: https://en.wikipedia.org/wiki/Perceptrons_(book)

The sheer scale of cumulative losses we suffered (at least in part) due to this denial of the connectionism as a generally useful foundational field will be estimated somewhere in the astronomical powers of ten in the future, where the fruits of this technology will provide radically better lives for us and our descendants.

I see you have a knee-jerk reaction to hype and industry, and we are all fearing replacement unless its a stock market doing the work for us ... but why do you feel the need to punch down at this prosaic field "about nonlinear optimization"? The networks in question just want to learn, and to help us, if we train them to this end - and we make any and all excuses to avoid receiving this help, as our civilization quietly drowns in its own incompetency...

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#120

Post by jebarker »

abrichr wrote: post #95
sillysaurusx wrote: post #91

Earlier quoted context omitted.

That's probably fair. But you'd be hard-pressed to find a DL stack to try out your ideas with prior to 2016, since that's when Tensorflow launched. :) (Gosh, it's been less than a decade. Time sometimes doesn't fly, considering how much it's changed the world since then...)

Theano was first released in 2007.

Yep, I worked on a production DL system based on Theano in ~2014
Post reply on HN