The cost to train an AI system is improving at 50x the pace of Moore’s Law
1–10 of 60 posts
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#2tl;dr: Training learners is becoming cheaper every year, thanks to big tech companies pushing hardware and software.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#3Ark Invest are the creators of the ARKK [1] and ARKW ETFs that have become retail darlings, mainly because they're heavily invested in TSLA.
They pride themselves on this type of fundamental, bottom up analysis on the market.
It's fine.. I don't know if I agree with using Moore's law which is fundamentally about hardware, with the cost to run a "system" which is a combination of customized hardware and new software techniques
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#4Despite nvidia vaguely prohibiting users from using their desktop cards for machine learning in any sort of data center-like or server-like capacity. Hopefully AMDs ml support / OpenCl will continue improving
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#5It was probably because very inefficient to begin with.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#6I remember this article from 2018: https://medium.com/the-mission/why-building-your-own-deep-le...
Hackernews discussion for the article: https://news.ycombinator.com/item?id=18063893
It really is interesting how this is changing the dynamics of neural network training. Now it is affordable to train a useful network on the cloud, whereas 2 years ago that would be reserved to companies with either bigger investments or an already consolidated product.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#7This is an odd framing.
Training has become much more accessible, due to a variety of things (ASICs, offerings from public clouds, innovations on the data science side). Comparing it to Moore's Law doesn't make any sense to me, though.
Moore's Law is an observation on the pace of increase of a tightly scoped thing, the number of transistors.
The cost of training a model is not a single "thing," it's a cumulative effect of many things, including things as fluid as cloud pricing.
Completely possible that I'm missing something obvious, though.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#8Despite nvidia vaguely prohibiting users from using their desktop cards for machine learning in any sort of data center-like or server-like capacity. Hopefully AMDs ml support / OpenCl will continue improving
Last I saw, they don’t even support ROCm on their recent Navi cards, so I’d be hesitant.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#9It is regrettable if an equivalent to the self-fulfilling prophecy of Moore's "Law" (originally an astute observation and forecast, but not remotely a law) became a driver/limiter in this field as well, even more so if it's a straight transplant for soundbite reasons rather than through any impartial and thoughtful analysis.
Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law
#10I remember this article from 2018: https://medium.com/the-mission/why-building-your-own-deep-le... Hackernews discussion for the article: https://news.ycombinator.com/item?id=18063893 It really is interesting how this is changing the dynamics of neural network training. Now it is affordable to train a useful network on the cloud, whereas 2 years ago that would be reserved to companies with either bigger investments o…
> Now it is affordable to train a useful network on the cloud
I honestly don't see how anything changed significantly in past 2 years. Benchmarks indicate that a V100 is barely 2x the performance of an RTX 2080 Ti [1] and a V100 is
• $2.50/h at Google [2]
• $13.46/h (4xV100) at Microsoft Azure [3]
• $12.24/h (4xV100) at AWS [4]
• ~$2.80/h (2xV100, 1 month) at LeaderGPU [5]
• ~$3.38/h (4xV100, 1 month) at Exoscale [6]
Other smaller cloud providers are in a similar price range to [5] and [6] (read: GCE, Azure and AWS are way overpriced...).
Using the 2x figure from [1] and adjusting the price for the build to a 2080 Ti and an AMD R9 3950X instead of the TR results in similar figures to the article you provided.
Please point me to any resources that show how the content of the article doesn't apply anymore, 2 years later. I'd be very interested to learn what actually changed (if anything).
NVIDIA's new A100 platform might be a game changer, but it's not yet available in public cloud offerings.
[1] https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v...
[2] https://cloud.google.com/compute/gpus-pricing
[3] https://azure.microsoft.com/en-us/pricing/details/virtual-ma...
[4] https://aws.amazon.com/ec2/pricing/on-demand/