Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The cost to train an AI system is improving at 50x the pace of Moore’s Law

ark-invest.com

Post reply on HN
Print view View original

41–50 of 60 posts

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#41

Post by Barrin92 »

jacquesm wrote: post #32
lukevp wrote: post #31

What are some domains that a solo developer could build something commercially compelling to capture some of this $37 trillion? Are there any workflows or tools or efficiencies that could be easily realized as a commercial offering that would not require massive man hours to implement?

Take any domain that requires classification work that has not yet been targeted and make a run for it. You likely will be able to adapt one of the existing nets or even use transfer learning to outperform a human. That's the low hanging fruit. For instance: quality control: abnormality detection (for instance: in medicine), agriculture (lots of movement there right now), parts inspection, assembly inspection, sortin…

> abnormality detection (for instance: in medicine), agriculture (lots of movement there right now), parts inspection, assembly inspection, sorting and so on

none of these is anything someone can run from their bedroom because they have very high quality and regulatory requirements and require constant work outside of the actual AI training.

This is actually reflected in the margins of "AI" companies, which are significantly lower than traditional SAAS businesses and require significantly more manpower to deal with the long tailed problems, which is where the AI fails but it's what actually matters.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#42

Post by op03 »

lukevp wrote: post #31

What are some domains that a solo developer could build something commercially compelling to capture some of this $37 trillion? Are there any workflows or tools or efficiencies that could be easily realized as a commercial offering that would not require massive man hours to implement?

Extracting and selling data stuck in the mountain ranges of pdfs and other useless formats in every large corp, org, govt dept on the planet.

Do it for a couple publicly available docs and then contact the org saying you offer 'archive digitization' so their data ppl can mine for intelligence.

Most of the time and resources of 'Digital Transformation'/Data Science Depts goes to just manually extracting info from all kinds of old docs, pdfs, spreadsheets containing institutional knowledge.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#43

Post by jacquesm »

Barrin92 wrote: post #41
jacquesm wrote: post #32

Earlier quoted context omitted.

Take any domain that requires classification work that has not yet been targeted and make a run for it. You likely will be able to adapt one of the existing nets or even use transfer learning to outperform a human. That's the low hanging fruit. For instance: quality control: abnormality detection (for instance: in medicine), agriculture (lots of movement there right now), parts inspection, assembly inspection, sortin…

> abnormality detection (for instance: in medicine), agriculture (lots of movement there right now), parts inspection, assembly inspection, sorting and so on none of these is anything someone can run from their bedroom because they have very high quality and regulatory requirements and require constant work outside of the actual AI training. This is actually reflected in the margins of "AI" companies, which are signi…

Well, depending on the size of your bedroom ;) I've seen teams of two people running fairly impressive ML based stuff. They were good enough at it that they didn't remain at two people for very long but that was more than enough to be useful to others. One interesting company - that I'm free to talk about - did a nice one on e-commerce sites to help with risk management: spot fraudulent orders before they ship.

In the long term, and to stay competitive you will always have to get out of bed and go to work. But the initial push can easily be just a very low number of people engaging an otherwise dormant niche.

Yes, medicine has regulatory requirements. But as long as you advise rather than diagnose the regulatory requirements drop to almost nil.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#44

Post by Isinlor »

lukevp wrote: post #31

What are some domains that a solo developer could build something commercially compelling to capture some of this $37 trillion? Are there any workflows or tools or efficiencies that could be easily realized as a commercial offering that would not require massive man hours to implement?

You need to be creative. But one example - colorizing old photos: https://twitter.com/citnaj

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#45

Post by vertak »

lukevp wrote: post #31

What are some domains that a solo developer could build something commercially compelling to capture some of this $37 trillion? Are there any workflows or tools or efficiencies that could be easily realized as a commercial offering that would not require massive man hours to implement?

You can give this article by Chip Huyen a read. Mayhaps you will find a niche for a solo or small dev team. Though it is focused on MLOps if that makes a different for the type of niche you're looking for.

https://huyenchip.com/2020/06/22/mlops.html

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#46

Post by qayxc »

gchamonlive wrote: post #36
qayxc wrote: post #27

Earlier quoted context omitted.

But that's not what the actual data says. Here's some figures from an actual benchmark [1] w.r.t. training costs: 1. [Mar 2020] $7.43 (AlibabaCloud, 8xV100, TF v2.1) 2. [Sep 2018] $12.60 (Google, 8 TPU cores, TF v1.11) 3. [Mar 2020] $14.42 (AlibabaCloud, 128xV100, TF v2.1) -- Training time didn't go down exponentially either [1]: 1. [Mar 2020] 0:02:38 (AlibabaCloud, 128 x V100, TF v2.1) 2. [May 2019] 0:02:43 (Huawei…

I guess I should have been more skeptical of the articles figures. But still, if we give the benefit of the doubt, is there any scenario we might see the reduction mentioned? 1000 to 10 USD?

The scenario is indeed there - if you take early 2017 numbers and restrict yourself to AWS/Google/Azure and outdated hardware and software, you can get to the US$1000 figure.

Likewise, if your other point of comparison is late 2019 AlibabaCloud spot pricing, you can get to US$10 for the same task.

Realistically, though, that's worst case 2017 vs best case 2019/2020. So you sure, you can get to that if you choose your numbers correctly.

They basically compared results from H/W that even in 2017 was 2 generations behind with the latest H/W. So yeah - between 2015 and 2019 we indeed saw a cost reduction from ~1000 to ~10 USD (on the major cloud provider vs best offer today scale).

I only take issue with the assumption that the trend continues this way, which it doesn't seem to.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#48

Post by KorfmannArno »

lukevp wrote: post #31

What are some domains that a solo developer could build something commercially compelling to capture some of this $37 trillion? Are there any workflows or tools or efficiencies that could be easily realized as a commercial offering that would not require massive man hours to implement?

krisp.ai but using gpu (also on mac) and with desktop version for ubuntu linux.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#49

Post by sdenton4 »

solidasparagus wrote: post #12

Resnet-50 with DawnBench settings is a very poor choice for illustrating this trend. The main technique driving this reduction in cost-to-train has been finding arcane, fast training schedules. This sounds good until you realize its a type of sleight of hand where finding that schedule takes tens of thousands of dollars (usually more) that isn't counted in cost-to-train, but is a real-world cost you would experience…

I would guess that this means DawnBench is basically working. You'll get some "overfit" training schedule optimizations, but hopefully amongst those you'll end up with some improvements you can take to other models.

We also seem to be moving more towards a world where big problem-specific models are shared (BERT, GPT), so that the base time to train doesn't matter much unless you're doing model architecture research. For most end-use cases in language and perception, you'll end up picking up a 99%-trained model, and fine tuning on your particular version of the problem.

Re: The cost to train an AI system is improving at 50x the pace of Moore’s Law

#50

Post by smabie »

yelloweyes wrote: post #37
jacquesm wrote: post #32

Earlier quoted context omitted.

Take any domain that requires classification work that has not yet been targeted and make a run for it. You likely will be able to adapt one of the existing nets or even use transfer learning to outperform a human. That's the low hanging fruit. For instance: quality control: abnormality detection (for instance: in medicine), agriculture (lots of movement there right now), parts inspection, assembly inspection, sortin…

anything that's even remotely profitable is already taken

Ah yes, of course. There will never be A new profitable ML startup until the end of time. Makes perfect sense.
Post reply on HN