Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The post-exponential era of AI and Moore’s Law

techcrunch.com

Post reply on HN
Print view View original

51–60 of 150 posts

Re: The post-exponential era of AI and Moore’s Law

#51

Post by FartyMcFarter »

pmoriarty wrote: post #27
xyproto wrote: post #21

Earlier quoted context omitted.

Yet AI beat humans at every strategic game out there, from Chess and Go to StarCraft2. We don't have general AI, but in some fields, human understanding have already been outpaced.

Those three games are very far from "every strategic game out there", and even they can be trivially made much more difficult for AI's by growing their board size, for example, making more types of pieces with different moves, or making other changes to their rules. There are hundreds of chess variants alone, for instance.

> growing their board size, for example, making more types of pieces with different moves, or making other changes to their rules

This would also make the games more difficult for humans, and it would erase much of the accumulated knowledge about strategy.

Re: The post-exponential era of AI and Moore’s Law

#52

Post by perlpimp »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

what a great article. would give one a way to have introspection of how and why these methods work and going forward learning machine learning/AI.

Re: The post-exponential era of AI and Moore’s Law

#53

Post by TomMarius »

mikorym wrote: post #44
boyadjian wrote: post #42

Maybe it is a good thing, so the singularity will be avoided. There is so much wrong usages of AI.

Mathematician here, I don't think the singularity as popularised through people like Ray Kurzweil would happen. In particular, their metaphor of an "infinity" point seems to me like the incorrect application of mathematical ideas to social contexts. I think the question of artificial general intelligence is a different question than a singularity, much like how the Chinese room addresses a different question than the…

Why is it a different question? AFAIK the assumption is that if the computer can improve itself (which it can if it has IQ 100), there is nothing holding the singularity back.

Re: The post-exponential era of AI and Moore’s Law

#54

Post by ilaksh »

General purpose AI is not waiting for more compute power. Its waiting for algorithms that can do it.

I'm not 100% sure deep learning can get all the way there. We might need a new approach. But the deep learning leaders are taking it very seriously and making some progress.

For example, Yann LeCun is talking about self-supervised, learning models of the world. MILA (Bengio's group) is talking about "state representation learning, or the ability to capture latent generative factors of an environment". Hinton now has capsule networks. In my opinion these types of approaches are very promising for general purpose AI.

Re: The post-exponential era of AI and Moore’s Law

#55

Post by p1esk »

pmoriarty wrote: post #28
networkimprov wrote: post #25

Earlier quoted context omitted.

The heck with all that. Show me the software that beats Roger, Rafa, and Nole at tennis!

I'm not sure if you'll have that long to wait, considering the kind of success Boston Robotics has been having. I would be surprised if by the end of the century robots couldn't beat humans at every physical sport in existence. It's likely that robots and robotically-enhanced humans will both be banned from competing directly with humans in sports, because it'll be no contest -- in the robots' favor.

Ping pong is almost solved: https://www.youtube.com/watch?v=kZzL2rDNSJk Give it a year or two and it will probably beat the world champion. Tennis is more challenging, but still, the surface is flat, no obstacles, so a platform with wheels carrying an arm with a racket could be fast and stable.

I'm not even sure what would be the hardest part to make a champion tennis robot.

Re: The post-exponential era of AI and Moore’s Law

#56

Post by sprash »

I predict there will be much more assembly programming required in the future to squeeze out as much as performance as possible because the end of Moore's Law is already very apparent for several applications that can not be easily parallelized. It is a complete myth that you "can't beat the C-compiler" as it is claimed so often. The compiler can't know many things you know about the problem at hand. So far I have been able to beat gcc on a regular basis while just relying on x86_64 (which means using no instructions/registers beyond SSE2.0).

Right now I'm looking for a C/Shader-like language for x86_64/Linux which allows you to be much closer to the metal without requiring you to go down to the cumbersome level of ASM syntax and at the same time shedding libc and use syscalls instead (E.g. if you have specific knowledge about the data structures you want to map you can use the heap much more efficient than plain old malloc). So far I found nothing.

Re: The post-exponential era of AI and Moore’s Law

#57

Post by brazzy »

carlmr wrote: post #5
ChuckMcM wrote: post #2

The article discusses one of the impacts of compute performance growth rates slowing while compute demand (training AI models) is growing exponentially. The 'end of Moore's law' (as a measure of performance, not density) is probably the most significant "Tech" story of the next decade. Why? Because it is going to demand engineers who can write fast code over engineers who write code fast. A lot of frameworks and abst…

>Because it is going to demand engineers who can write fast code over engineers who write code fast. I surely hope so, the waste mentality has to end.

Premature optimization will never become the right thing to do. Most code has absolutely no reason to be particularly fast because it doesn't get executed very frequently.

Re: The post-exponential era of AI and Moore’s Law

#58

Post by csomar »

cft wrote: post #3
ChuckMcM wrote: post #2

The article discusses one of the impacts of compute performance growth rates slowing while compute demand (training AI models) is growing exponentially. The 'end of Moore's law' (as a measure of performance, not density) is probably the most significant "Tech" story of the next decade. Why? Because it is going to demand engineers who can write fast code over engineers who write code fast. A lot of frameworks and abst…

Rust will fare well. Ruby won't.

That has a lot to do with why I took on learning Rust. My last year Macbook Pro has a 6-core CPU. At this rate, we should have 32-64-core CPUs for consumers in a few years. Not using all the processors means your program only makes use of 3-1.5% of the available power.

My prediction is that, in a few years, lots of programming languages and software developers will simply become obsolete because what the other guys can produce is heaps of performance, security and manageability atop.

But I might be wrong, so make your career decisions carefully.

Re: The post-exponential era of AI and Moore’s Law

#59

Post by YeGoblynQueenne »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

Great read. To follow up also read a comment on that piece titled "A better lesson" by Rodney Brooks:

https://rodneybrooks.com/a-better-lesson/

Re: The post-exponential era of AI and Moore’s Law

#60

Post by carlmr »

brazzy wrote: post #57
carlmr wrote: post #5

Earlier quoted context omitted.

>Because it is going to demand engineers who can write fast code over engineers who write code fast. I surely hope so, the waste mentality has to end.

Premature optimization will never become the right thing to do. Most code has absolutely no reason to be particularly fast because it doesn't get executed very frequently.

Nobody said that. Just that you should think about which algorithms you use, profile, and not waste so much. The wasted power at this point is significant.
Post reply on HN