Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The post-exponential era of AI and Moore’s Law

techcrunch.com

Post reply on HN
Print view View original

101–110 of 150 posts

Re: The post-exponential era of AI and Moore’s Law

#101

Post by giacaglia »

hyperpallium wrote: post #88

I am optimistic that Moore's Law will eventually recover, with a new technology, perhaps silicon-based, perhaps not. Information processing is not intrinsically limited by silicon - for example, mammalian brains are more powerful. But that's lomg-term, big-picture. Technologies can remain stagnant longer than you can remain alive.

It's funny that the article mentions that processors have not increase their speed, and therefore Moore's law is dead. In fact, as many people know Intel has been struggling with their 10nm chip, and now with a new CEO, this might change. All other processor manufacturers are catching up and some are moving ahead of Intel. That's the whole reason Apple is trying to move away from X86 with their MACs

Re: The post-exponential era of AI and Moore’s Law

#102

Post by TheOtherHobbes »

TomMarius wrote: post #80
TheOtherHobbes wrote: post #72

Earlier quoted context omitted.

Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition. It's the difference between a computer that is taught how to compose okay-ish music, and a computer that learns spontaneously how to compose really really great music and do all of the social, cultural, and financial things requi…

> Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition. I disagree. A program improves itself when it reacts to a problem and implements a solution. Obviously that is very general, but enough. A human of IQ 100 certainly can develop software; a program of IQ 100 should be able to do…

I don't mean to be rude, but I'm impressed that you seem to have completely skated over the content of my comment.

Please read it again. You need to understand what a "problem" and a "solution" both are - in detail - because otherwise you have nothing to work with.

And a human of IQ 100 will only ever develop poor software. If you scale horizontally, you won't get game-changers - you'll just get a flood of equally poor software more quickly.

Re: The post-exponential era of AI and Moore’s Law

#103

Post by csomar »

oblio wrote: post #91
csomar wrote: post #58

Earlier quoted context omitted.

That has a lot to do with why I took on learning Rust. My last year Macbook Pro has a 6-core CPU. At this rate, we should have 32-64-core CPUs for consumers in a few years. Not using all the processors means your program only makes use of 3-1.5% of the available power. My prediction is that, in a few years, lots of programming languages and software developers will simply become obsolete because what the other guys c…

I hope you're right. However Intel Core Duo was released in January 2006 and it had 2 cores. So in a bit more than 13 years we only have 3x as many cores. Core 2 Quad was released in January 2007, so if you count from that one, a bit more than 12 years for 1.5x as many cores. I wouldn't bet on widespread 64 core CPUs for consumers in the next 10 years or so. Most commercial software just doesn't use it efficiently an…

> However Intel Core Duo was released in January 2006 and it had 2 cores. So in a bit more than 13 years we only have 3x as many cores. Core 2 Quad was released in January 2007, so if you count from that one, a bit more than 12 years for 1.5x as many cores.

https://xkcd.com/605/

If you are going to extrapolate, don't pick the date. Take all the numbers you have and trace the average value. If you are picking the numbers, you can extrapolate to any point you want.

> I wouldn't bet on widespread 64 core CPUs for consumers in the next 10 years or so.

We already have 32-core processors for servers. Imo, the biggest challenge will be the laptop and heat management rather than putting a bunch of processors together.

> We need a software breakthrough, a killer app, something that people really want and that really needs a ton of cores.

We need that now. Most software I'm using is slow. Part of why I picked the iPhone (and I bet many people did) was speed. It is fast. People care about speed and if you introduce them to something faster then they'll get hooked. Also, you are already using all of your cores because of multi-tasking. It makes your computer experience overall better. At 6 cores, multi-threading for an application doesn't give much noticeable experience, since you probably have 5-6 things running on your computer doing other stuff like network or an audio-player.

> For now, I don't see it. But then again, that's how killer apps are, nobody sees them until they're there.

> I hope you're right.

Don't worry, just hang in there. The start of multi-cores was slow but then that's all there is right now. Seems intel/amd have mastered how to add more cores but the CPU frequency is pretty much capped going forward!

Re: The post-exponential era of AI and Moore’s Law

#104

Post by Veedrac »

_bxg1 wrote: post #14
m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

It could be that the slowing of Moore's Law results in a more diverse array of specialized computing hardware. After all, the resurgence of ML didn't happen because x86 got fast enough, but because video games - of all things - funded the maturation of a whole new category of massively-parallel chips.

Moore's law never slowed. At worst, one can rightly say Intel had issues with 10nm.

https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrh...

Re: The post-exponential era of AI and Moore’s Law

#105

Post by Veedrac »

KKKKkkkk1 wrote: post #83
m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

Sutton argues that algorithm development needs to be shaped by the assumption that compute power will continue growing exponentially into the future. At this point, it is commonly believed that this is not the case. One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation become…

I'm fairly sure the consensus from the hardware side, especially considering how companies like NVIDIA, Intel and Arm are reacting, is that we still have a lot of exponential growth to go, especially for AI. A factor 1000 increase in model sizes seems within reason within 10ish years; I go over some arguments in the link below.

https://www.reddit.com/r/MachineLearning/comments/ds1xvc/d_d...

Re: The post-exponential era of AI and Moore’s Law

#106

Post by ThrowawayR2 »

TomMarius wrote: post #80
TheOtherHobbes wrote: post #72

Earlier quoted context omitted.

Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition. It's the difference between a computer that is taught how to compose okay-ish music, and a computer that learns spontaneously how to compose really really great music and do all of the social, cultural, and financial things requi…

> Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition. I disagree. A program improves itself when it reacts to a problem and implements a solution. Obviously that is very general, but enough. A human of IQ 100 certainly can develop software; a program of IQ 100 should be able to do…

Many here have an IQ well over 100 and have no idea how to rebuild an improved version of themselves or even how their consciousness works internally. Seems rather likely any sapient AI we cook up will have just as little idea how itself works as we do how we work. QED, no singularity.

Re: The post-exponential era of AI and Moore’s Law

#107

Post by buboard »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

thats great arguments except vision. Knowledge about the physiology of the visual system DID go into designing convolutional networks. That probably doesn't extend to other domains though, and things prove once again that our intuitions may be misleading.

Re: The post-exponential era of AI and Moore’s Law

#108

Post by _bxg1 »

xyproto wrote: post #21
_bxg1 wrote: post #6

Honestly, this is a bit of a relief. All of the AI nightmare scenarios (be they the Terminator kind, or the more realistic hyper-empowerment-of-a-few-elites kind) rely on that exponential growth continuing unimpeded. If there's no exponential growth, there's no runaway AI that swiftly outpaces human understanding.

Yet AI beat humans at every strategic game out there, from Chess and Go to StarCraft2. We don't have general AI, but in some fields, human understanding have already been outpaced.

Each of those has fixed rules, and each took lots of effort from human researchers to hand-craft and hand-train the models. Not that it isn't impressive, but it's still fundamentally a different thing than general intelligence. I don't think it's just a matter of doing more of the same.

Re: The post-exponential era of AI and Moore’s Law

#109

Post by AnimalMuppet »

p1esk wrote: post #55
pmoriarty wrote: post #28

Earlier quoted context omitted.

I'm not sure if you'll have that long to wait, considering the kind of success Boston Robotics has been having. I would be surprised if by the end of the century robots couldn't beat humans at every physical sport in existence. It's likely that robots and robotically-enhanced humans will both be banned from competing directly with humans in sports, because it'll be no contest -- in the robots' favor.

Ping pong is almost solved: https://www.youtube.com/watch?v=kZzL2rDNSJk Give it a year or two and it will probably beat the world champion. Tennis is more challenging, but still, the surface is flat, no obstacles, so a platform with wheels carrying an arm with a racket could be fast and stable. I'm not even sure what would be the hardest part to make a champion tennis robot.

The hardest part might be to create one that can store enough energy to last through a five-hour match.

Re: The post-exponential era of AI and Moore’s Law

#110

Post by pmoriarty »

AnimalMuppet wrote: post #109
p1esk wrote: post #55

Earlier quoted context omitted.

Ping pong is almost solved: https://www.youtube.com/watch?v=kZzL2rDNSJk Give it a year or two and it will probably beat the world champion. Tennis is more challenging, but still, the surface is flat, no obstacles, so a platform with wheels carrying an arm with a racket could be fast and stable. I'm not even sure what would be the hardest part to make a champion tennis robot.

The hardest part might be to create one that can store enough energy to last through a five-hour match.

Why couldn't it just change batteries as needed, or just be wired up to power for that matter?

I'd expect something like wrestling would be much more challenging than non-contact sports, assuming, of course, that the robot's strength and torque was limited to human levels.

Wrestling is actually a pretty interesting case, now that I think about it for a bit, as robots wouldn't need to be limited to human-like bodies. In principle, they could have, say, a starfish-like or octopus-like body, where its tentacles would allow it to grip its human opponent in ways that may be impossible to escape, even were the robot limited to human strength.

So I suspect that in sports like wrestling (and maybe all other sports), robots that are allowed to compete would likely be limited to human-like forms, as other forms could give them too great an advantage.

Post reply on HN