Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The post-exponential era of AI and Moore’s Law

techcrunch.com

Post reply on HN
Print view View original

41–50 of 150 posts

Re: The post-exponential era of AI and Moore’s Law

#41

Post by jayd16 »

sgt101 wrote: post #37
npo9 wrote: post #19

Earlier quoted context omitted.

Let me fuel your paranoia a little. It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Quantum computers might play a role in increasing computational power in the next decade. Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the adv…

>cloud computing will continue to see decreases in costs cloud computing costs much more than on prem; it's available on demand, which is why it's attractive to some companies. For everyone else it's an outsourcing play that unlocks capex for revenue generation. >Quantum computers might play a role in increasing computational power in the next decade. Nope. No chance. Best case is that QC will provide improved molecu…

Its not that computers in the cloud are cheap, its that the capacity will continue to grow and they're fully networked such that our AI overlords can take full advantage of them.

Re: The post-exponential era of AI and Moore’s Law

#44

Post by mikorym »

boyadjian wrote: post #42

Maybe it is a good thing, so the singularity will be avoided. There is so much wrong usages of AI.

Mathematician here, I don't think the singularity as popularised through people like Ray Kurzweil would happen.

In particular, their metaphor of an "infinity" point seems to me like the incorrect application of mathematical ideas to social contexts.

I think the question of artificial general intelligence is a different question than a singularity, much like how the Chinese room addresses a different question than the Turing test.

Re: The post-exponential era of AI and Moore’s Law

#45

Post by ivalm »

xyproto wrote: post #21
_bxg1 wrote: post #6

Honestly, this is a bit of a relief. All of the AI nightmare scenarios (be they the Terminator kind, or the more realistic hyper-empowerment-of-a-few-elites kind) rely on that exponential growth continuing unimpeded. If there's no exponential growth, there's no runaway AI that swiftly outpaces human understanding.

Yet AI beat humans at every strategic game out there, from Chess and Go to StarCraft2. We don't have general AI, but in some fields, human understanding have already been outpaced.

In complete information games like chess and go the AI does reign supreme. In SC2, the AI plays highly abusable strategies that would not win with proper prep vs top players. Same thing with Dota bots by OpenAI.

Re: The post-exponential era of AI and Moore’s Law

#46

Post by adwn »

codesushi42 wrote: post #23
_bxg1 wrote: post #14

Earlier quoted context omitted.

It could be that the slowing of Moore's Law results in a more diverse array of specialized computing hardware. After all, the resurgence of ML didn't happen because x86 got fast enough, but because video games - of all things - funded the maturation of a whole new category of massively-parallel chips.

the resurgence of ML ... because video games - of all things - funded the maturation of a whole new category of massively-parallel chips Fake news. It has far more to do with the rise of distributed computing than the existence of GPUs.

Don't call a statement you disagree with "fake news". It a) doesn't even make sense in this case, and b) distracts from the discussion.

Re: The post-exponential era of AI and Moore’s Law

#47

Post by dogma1138 »

npo9 wrote: post #32
martinpw wrote: post #26

Earlier quoted context omitted.

> It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Why would cloud computing continue to see significant decreases in costs once Moore's law stops? Clearly there will be some ongoing decline in infrastructure costs, but if your main requirement is for lots of compute (for AI) then it seems har…

I was speaking about linear decreases. Cheaper power, more refined manufacturing processes, and economy of scale benefits are where I expect some of the linear benefits to come from. We can also expect CPU manufactures to compete for awhile, getting increasingly marginal improvements. Linear improvements will get us there in the end.

Why would power will become cheaper? Also manufacturing refinement? Sure it happens but not to any meaningful degree these days it’s more about bribing a new process to the same cost/efficiency as your older manufacturing processes.

As far as economy of scales goes it’s also not a given.

Large cloud providers are much more adapt at reusing old hardware for their PAAS and weaker instances.

I’ve had a chance to talk to a few reps from Dell once and while cloud buy a lot of servers they also upgrade them much less often than large enterprises that run their own DC this is because Amazon can always continue to make money on 3-5 year old hardware by shifting it off to cheaper tiers or by running things like SQS or Lambdas on it in the background.

On the other hand if you have a 3-5 year old server and you need more horse power if you manage your own DC or rack space the hosting costs make it economical to keep underperforming hardware.

Not to mention other costs such as support are quite different, if an Amazon server dies they don’t care. Traditional self hosted environments will require extended support because they can’t have the same redundancy as Amazon.

If you delegate a 5 year old server to some back office applications while you use your new ones for client facing apps the old server is still a mission critical box even if it doesn’t directly run revenue generating services.

Basically Amazon buys to grow, on-prem orgs buy to both grow and upgrade.

If you have an organization the size of Amazon that is completely self hosted they will end up buying much more hardware over a decade than Amazon.

Re: The post-exponential era of AI and Moore’s Law

#48

Post by TomMarius »

sgt101 wrote: post #37
npo9 wrote: post #19

Earlier quoted context omitted.

Let me fuel your paranoia a little. It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Quantum computers might play a role in increasing computational power in the next decade. Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the adv…

>cloud computing will continue to see decreases in costs cloud computing costs much more than on prem; it's available on demand, which is why it's attractive to some companies. For everyone else it's an outsourcing play that unlocks capex for revenue generation. >Quantum computers might play a role in increasing computational power in the next decade. Nope. No chance. Best case is that QC will provide improved molecu…

> cloud computing costs much more than on prem;

Yeah, and that will continue to go down, exactly because of what you stated in the next sentence.

Re: The post-exponential era of AI and Moore’s Law

#49

Post by ben_w »

pmoriarty wrote: post #33
ChuckMcM wrote: post #2

The article discusses one of the impacts of compute performance growth rates slowing while compute demand (training AI models) is growing exponentially. The 'end of Moore's law' (as a measure of performance, not density) is probably the most significant "Tech" story of the next decade. Why? Because it is going to demand engineers who can write fast code over engineers who write code fast. A lot of frameworks and abst…

It's not just training AI models that's demanding exponentially increasing compute power, but also streaming, storing, and processing the enormous amounts of scientific data coming from (for example) high-end physics experiments at world-class laboratories like CERN or the LHC, and there's also the tremendous amounts of surveillance data and data processing that's sprung up in recent decades, with an insatiable appet…

True, but from what people were telling me before I left university, scientific processing always produced more data than could be analysed, even after accounting for the effort put into efficient algorithms.

Re: The post-exponential era of AI and Moore’s Law

#50

Post by ben_w »

tsimionescu wrote: post #24
xyproto wrote: post #21

Earlier quoted context omitted.

Yet AI beat humans at every strategic game out there, from Chess and Go to StarCraft2. We don't have general AI, but in some fields, human understanding have already been outpaced.

Specialized fields like games aren't necessarily a problem - computers have been beating us at arithmetic for decades, and it isn't exactly world shattering. Also, a minor nitpick: AI has not succeeded at beating the best players at SC2, though it did beat the vast majority of the ladder.

> Specialized fields like games aren't necessarily a problem - computers have been beating us at arithmetic for decades, and it isn't exactly world shattering.

I would argue that it is, because that’s the driver for industrial automation.

Post reply on HN