Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The post-exponential era of AI and Moore’s Law

techcrunch.com

Post reply on HN
Print view View original

11–20 of 150 posts

Re: The post-exponential era of AI and Moore’s Law

#11

Post by foota »

ChuckMcM wrote: post #2

The article discusses one of the impacts of compute performance growth rates slowing while compute demand (training AI models) is growing exponentially. The 'end of Moore's law' (as a measure of performance, not density) is probably the most significant "Tech" story of the next decade. Why? Because it is going to demand engineers who can write fast code over engineers who write code fast. A lot of frameworks and abst…

I'm not sure that's a given, who's to say we won't be happy with the current performance we have?

Re: The post-exponential era of AI and Moore’s Law

#14

Post by _bxg1 »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

It could be that the slowing of Moore's Law results in a more diverse array of specialized computing hardware. After all, the resurgence of ML didn't happen because x86 got fast enough, but because video games - of all things - funded the maturation of a whole new category of massively-parallel chips.

Re: The post-exponential era of AI and Moore’s Law

#16

Post by yarg »

sdoken123 wrote: post #12

Could it be that with Quantum Computing computational power will continue to increase exponentially?

For some problems, perhaps - but it could also be the case maintaining entanglement becomes exponentially harder with an increasing number of qubits.

Re: The post-exponential era of AI and Moore’s Law

#17

Post by yarg »

npo9 wrote: post #13
sdoken123 wrote: post #12

Could it be that with Quantum Computing computational power will continue to increase exponentially?

Is Quantum Computing computational power increasing exponentially now?

Yes - and on two fronts. The number of qubits that can be entangled is currently going up at an exponential rate, and with each additional qubit the power of the machine (in a certain sense) doubles.

Re: The post-exponential era of AI and Moore’s Law

#18

Post by ChuckMcM »

m3at wrote: post #10

The "Bitter Lesson" post from Rich Sutton from earlier this year [1] seems a very good complement to this article: he explain how all of the big improvements in the field came from new methods that leveraged the much larger compute available from Moore's law, instead of progressive buildup over existing methods. A great quote from McCarthy also regularly referenced by Sutton is "Intelligence is the computational part…

Thank you for that, it was excellent.

Re: The post-exponential era of AI and Moore’s Law

#19

Post by npo9 »

_bxg1 wrote: post #6

Honestly, this is a bit of a relief. All of the AI nightmare scenarios (be they the Terminator kind, or the more realistic hyper-empowerment-of-a-few-elites kind) rely on that exponential growth continuing unimpeded. If there's no exponential growth, there's no runaway AI that swiftly outpaces human understanding.

Let me fuel your paranoia a little.

It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving.

Quantum computers might play a role in increasing computational power in the next decade.

Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the advances dry up maybe more fundamental research will produce much better results with our current technology.

Re: The post-exponential era of AI and Moore’s Law

#20

Post by _bxg1 »

npo9 wrote: post #19
_bxg1 wrote: post #6

Honestly, this is a bit of a relief. All of the AI nightmare scenarios (be they the Terminator kind, or the more realistic hyper-empowerment-of-a-few-elites kind) rely on that exponential growth continuing unimpeded. If there's no exponential growth, there's no runaway AI that swiftly outpaces human understanding.

Let me fuel your paranoia a little. It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Quantum computers might play a role in increasing computational power in the next decade. Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the adv…

Thanks :P
Post reply on HN