Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The post-exponential era of AI and Moore’s Law

techcrunch.com

Post reply on HN
Print view View original

81–90 of 150 posts

Re: The post-exponential era of AI and Moore’s Law

#81

Post by rrss »

Symmetry wrote: post #78

For a long time we had a situation where transistors got smaller and cheaper and faster and more power efficient all at the same through the wonders of Denard scaling and we gestured broadly at the whole thing and called it all "Moore's Law" without needing to distinguish which exponential improvement the term referred to. But in the mid 2000s Denard scaling broke down and now it looks like transistors are still gett…

Transistors haven't really been getting smaller for the last few nodes, they've mostly just been getting denser.

AFAIK nobody currently knows how make finfets much smaller without making them awful.

Re: The post-exponential era of AI and Moore’s Law

#82

Post by Shorel »

sprash wrote: post #56

I predict there will be much more assembly programming required in the future to squeeze out as much as performance as possible because the end of Moore's Law is already very apparent for several applications that can not be easily parallelized. It is a complete myth that you "can't beat the C-compiler" as it is claimed so often. The compiler can't know many things you know about the problem at hand. So far I have be…

"Right now I'm looking for a C/Shader-like language for x86_64/Linux"

You seem like the ideal candidate for writing such a language. And the book about it.

Re: The post-exponential era of AI and Moore’s Law

#83

Post by KKKKkkkk1 »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

Sutton argues that algorithm development needs to be shaped by the assumption that compute power will continue growing exponentially into the future. At this point, it is commonly believed that this is not the case.

One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

Re: The post-exponential era of AI and Moore’s Law

#84

Post by ghettoimp »

DennisP wrote: post #68
ivalm wrote: post #9

Earlier quoted context omitted.

I don't have a hat in the game but why won't Ruby do well or not well based on this? It's not like basic webapps need to be much more efficient, and no one was writing high performance compute in ruby anyways.

Looking over the techempower benchmarks, the fastest web frameworks are over fifty times faster than ruby/rails. If you can replace a hundred web servers with two, that matters.

Sure. But an awful lot of folks don't have a hundred web servers. If you're running something simple, you can pretty much use whatever you want.

Re: The post-exponential era of AI and Moore’s Law

#85

Post by marcosdumay »

Symmetry wrote: post #78

For a long time we had a situation where transistors got smaller and cheaper and faster and more power efficient all at the same through the wonders of Denard scaling and we gestured broadly at the whole thing and called it all "Moore's Law" without needing to distinguish which exponential improvement the term referred to. But in the mid 2000s Denard scaling broke down and now it looks like transistors are still gett…

Transistors haven't been getting cheaper either. New processes have a higher cost per transistor for a while.

Or better, as old processes get improved, transistors are getting either more compactly packed or cheaper, but not both. The expensive ones are getting a little bit faster too, but not in proportion to the reduced die size.

Re: The post-exponential era of AI and Moore’s Law

#87

Post by oblio »

skohan wrote: post #63

I predict that the slowing of Moore's law is going to make HPC and optimization in general a much more valuable skillset in the next few decades. In the past 15 years or so, we've been more or less happy to treat CPU cycles as a limitless resource, and as a result modern software stacks have a lot of fat in them. At the end of the era of free speed increases, trimming the fat is going to be a lot more important.

It depends where this is going to happen. Libraries, OSes, RDBMSes, etc. will have to be optimized for sure. But actual apps? There's way more app writers than library writes.

And most line-of-business apps don't need to be faster, they need to be easier to use and more stable.

Re: The post-exponential era of AI and Moore’s Law

#88

Post by hyperpallium »

I am optimistic that Moore's Law will eventually recover, with a new technology, perhaps silicon-based, perhaps not. Information processing is not intrinsically limited by silicon - for example, mammalian brains are more powerful.

But that's lomg-term, big-picture. Technologies can remain stagnant longer than you can remain alive.

Re: The post-exponential era of AI and Moore’s Law

#89

Post by oblio »

brudgers wrote: post #65
coldtea wrote: post #38

Earlier quoted context omitted.

> Why? Because it is going to demand engineers who can write fast code over engineers who write code fast. Or just making slower things more orthogonally parallelized and throwing more hardware units at the problem (as opposed to more powerful hardware or faster code). Most code by engineers "who write code fast" is not in performance critical domains...

I agree mostly. But how long can engineers throw more and more and exponentially more hardware at the phone that fits in my pocket?

If my experience with the IT world has taught me anything, you'd be surprised for how long :-)
Post reply on HN