Live data from Hacker News

On the Impossibility of Supersized Machines (2017)

arxiv.org

41–50 of 100 posts

Re: On the Impossibility of Supersized Machines (2017)

#41
post #22

Earlier quoted context omitted.

> Calculate how many transistors and watts of power you'd need to replicate that functionality. I'm curious as to your answer. Because if one's building a purpose-built analog computer for the task, my estimate is a few hundred transistors, a few thousand passives, and ... an absolutely trivial amount of power on modern process.

I'm curious how we're even going to manage 420,000 pixels' worth (60,000 ommatidia, approximately 7 pixels each) of input with only a few hundred transistors, let alone do vector analysis on it. But let's say we can. Let's say we need 320 transistors, which would be 20 transistors per pixel. That's pretending 99.7% of the seven thousand synapses each neuron has are useless for our purpose, but we'll do it. A chimp br…

> I'm curious how we're even going to manage 420,000 pixels' worth (60,000 ommatidia, approximately 7 pixels each) of input with only a few hundred transistors, let alone do vector analysis on it.

If you define the problem as importing 420,000 pixels, and target recognitions, and vector analysis, then you need a whole lot more computation than the organism uses. But presumably you're going to also get better results. We both know that's not exactly what's happening, I think.

That is, we know we can solve similar tracking problems with a whole lot less state.

> That's pretending 99.7% of the seven thousand synapses each neuron has are useless for our purpose

Not really... I think we can imagine a whole lot of passives / linear operations involved, along with the big nonlinear processes we need transistors for.

We're also assuming there's no net benefit to cognition that can happen using transistors, I'll note-- e.g. they have a ton of bandwidth compared to neurons, can be multiplexed more readily, etc....

> Humans have 86 billion neurons. Subtracting 22 gives us 64 billion, times 20 transistors per neuron gives us 1.28 trillion transistors.

So about half the number packed onto Cerebras WSE-2 today.

> even pretending we exactly understood how sapience worked in the first place.

This is the big problem.

Re: On the Impossibility of Supersized Machines (2017)

#42
post #35

Earlier quoted context omitted.

My guess is as good as any other layperson's, but I don't see much work being done for it, and no real good definition of what it is so we could plan how to create it. OTOH, we see specialized intelligences do all soft of superhuman feats, all the time, and more impressive abilities join these all the time. These, however, are not human-like intelligences. They aren't even bee-like. They are so alien we don't see "ge…

To me, the term "alien" connotes a level of capability much more interesting than what I've actually seen from most modern systems. But point taken. And I'd like to believe that you're right about it only being a matter of time and desire, but I do also worry about the possibility that we're actually on a different kind of exponential curve and will instead reach a point where we see diminishing returns.

I have no doubts there will be diminishing returns at some point, specially with narrow AIs, where increases in complexity and cost of training models will not be able to improve on what’s already good enough.

AI is a tool like many others, useful for some things and not for others.

Re: On the Impossibility of Supersized Machines (2017)

#43

In case anyone is wondering, we have made zero progress on anything even remotely resembling Artificial Intelligence. Zero . Unfortunately of course, the people who might have some of the skills needed to actually build such a thing (at the bricks and mortar level anyway), are nearly those people whose understanding of what intelligence actually is may be less than ideal. As a hint, it has nothing to do with passing…

Wolves definitely don't have language.

Still, they possess an undeniable degree of intelligence. They also have cultures, that is, forms of knowledge passed between generations by teaching, not genetically, and differing between packs.

I suspect that a robot as intelligent as a dog, but with an easier interface, would be a great help to humans.

OTOH, what currently is called "AI" is mostly deep learning, a very important part of cognition and perception. Without modern results in computer perception and low-level cognition and control, a "more general" AI would be blind, deaf, and paralyzed in the real world.

I suspect that the older approaches based on more supervised ways to construct cognitive functions have not born all the fruit they could, and may eventually help create an AI with better higher-level reasoning. They are just not in vogue now, so the best researchers and fattest grants are in deep learning and around. Also, the hardware may not be there yet.

(A similar thing happened to neural networks. The first, one-layer, neural network was the perceptron created in 1958 [1] The approach, while valid and constantly developed, did not see a real uptake until early 2010s, when incomparably better hardware finally became available.)

[1]: https://en.wikipedia.org/wiki/Perceptron

Re: On the Impossibility of Supersized Machines (2017)

#44
post #28
post #8

Makes me wonder: how's the HN community feeling these days about the actual plausibility / timeline of humans developing true AGI? Personally the more I learn about the current state of AI, and in comparison the way the human brain works, the more skeptical (and slightly disappointed) I tend to get.

The human brain is estimated at 2.5 Pb of storage [0]. Assuming a "Moore's Law" like behavior of storage price, so that price halves every 2-3 years and assuming we use storage as a proxy for the space, access speed and computational power, the time it will take to have a $1000 computer that has the storage capacity of the brain will be in the 10-16 year time horizon. This puts the timeline to about 2029-2035. [0] ht…

It is not hard to create a RAID array with 2.5 Pb capacity.

The trick of the human brain is that the "processing power" is enmeshed into the "memory", so the brain must have a colossal computational bandwidth, even with pretty slow neurons. I suppose that bandwidth is larger than that of most modern GPU / TPU clusters, which also don't feature anything comparable to 2.5Pb or RAM in their disposal.

The revolution should be mostly in the architecture, much like the deep learning evolution was enabled by GPUs.

Re: On the Impossibility of Supersized Machines (2017)

#45

In case anyone is wondering, we have made zero progress on anything even remotely resembling Artificial Intelligence. Zero . Unfortunately of course, the people who might have some of the skills needed to actually build such a thing (at the bricks and mortar level anyway), are nearly those people whose understanding of what intelligence actually is may be less than ideal. As a hint, it has nothing to do with passing…

If you showed somebody from 1921 a page of text produced by GPT-3, told them that it was written by a machine, and then told them that we'd made no progress towards artificial intelligence, they'd laugh in your face.

You can take from that what you will, but I suspect it will always seem as though we've made no progress, because anything we learn to emulate we necessarily understand well enough that it will no longer seem magical. I wouldn't put it past us to start thinking of humans as automata before we declare that machines can think.

Re: On the Impossibility of Supersized Machines (2017)

#46
post #14

Earlier quoted context omitted.

AGI is currently as likely as teleportation, time travel or warp drives. You can write a computer program to do just about anything. Artificial "General" intelligence is simply not a thing. We're not even making progress toward it.

We have natural “general” intelligence which appears to be generated by boring old chemical/thermal/electrical interactions. Why wouldn’t we be able to recreate that at some (IMO very far) point?

More than that: we have literally billions of examples of human-level intelligence right here on Earth. We have not a single example of teleportation, time travel, FTL, and other staples of not-very-science fiction.

Guess what is more likely to be implemented.

Re: On the Impossibility of Supersized Machines (2017)

#47
post #28
post #8

Makes me wonder: how's the HN community feeling these days about the actual plausibility / timeline of humans developing true AGI? Personally the more I learn about the current state of AI, and in comparison the way the human brain works, the more skeptical (and slightly disappointed) I tend to get.

The human brain is estimated at 2.5 Pb of storage [0]. Assuming a "Moore's Law" like behavior of storage price, so that price halves every 2-3 years and assuming we use storage as a proxy for the space, access speed and computational power, the time it will take to have a $1000 computer that has the storage capacity of the brain will be in the 10-16 year time horizon. This puts the timeline to about 2029-2035. [0] ht…

In the past 10 years, I think we had a 8x increase in easily available storage going from 1TB drives being a $100 to a 16TB drive being roughly $250. So I would have to say that your time scale is way too optimistic at best.

Re: On the Impossibility of Supersized Machines (2017)

#49
post #8

Makes me wonder: how's the HN community feeling these days about the actual plausibility / timeline of humans developing true AGI? Personally the more I learn about the current state of AI, and in comparison the way the human brain works, the more skeptical (and slightly disappointed) I tend to get.

Nobody expected anything like supremacy in Go any time soon and then all of a sudden it happened. Maybe AI stagnates for a long time bow, maybe forever, maybe a big breakthrough happens tomorrow. Nobody knows, anyone confidently asserting anything is being foolish.

Re: On the Impossibility of Supersized Machines (2017)

#50

In case anyone is wondering, we have made zero progress on anything even remotely resembling Artificial Intelligence. Zero . Unfortunately of course, the people who might have some of the skills needed to actually build such a thing (at the bricks and mortar level anyway), are nearly those people whose understanding of what intelligence actually is may be less than ideal. As a hint, it has nothing to do with passing…

If you showed somebody from 1921 a page of text produced by GPT-3, told them that it was written by a machine, and then told them that we'd made no progress towards artificial intelligence, they'd laugh in your face. You can take from that what you will, but I suspect it will always seem as though we've made no progress, because anything we learn to emulate we necessarily understand well enough that it will no longer…

> If you showed somebody from 1921 a page of text produced by GPT-3, told them that it was written by a machine, and then told them that we'd made no progress towards artificial intelligence, they'd laugh in your face.

You can actually do it. 100 year old people usually don't follow news on artificial intelligence, so they will act genuine.

Post reply on HN