Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do so many startups claim machine learning is their long game?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#61

Post by goatinaboat »

nostrademons wrote: post #45
papln wrote: post #27

Earlier quoted context omitted.

> data ownership and pipelines not ML. By your argument, the money is in data hoarding and brokering, and renting that data (with DRM) to ML outfits, not doing ML. Anyway, cleaning dirty data isn't execeptionally hard, it's just boring work; getting the raw data is the hard part.

This is an interesting observation that deserves to be addressed. I think in practice, what happens is that there's no easy transactional boundary that can keep the data ownership and ML in separate firms. The theory of the firm [1] states that firms arise when transaction costs are less than the economic inefficiencies of centralized resource allocation. There are some pretty heavy transaction costs between doing da…

easy transactional boundary that can keep the data ownership and ML in separate firms

There is. Data broking is a well established business model in finance. Every bank and hedge fund is running ML models on data licensed from a provider such as Reuter’s, S&P, etc there are dozens of such brokers.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#62

Post by Jommi »

mdbug wrote: post #55
thecolorblue wrote: post #52

I agree with other points here, but I also think nobody wants to wake up 2 years from now and be the only company that was not investing in machine learning. It could turn into nothing, or it could be 100x the time and money put in now. So, of the 4 outcomes: (1) buy in now + worthless (2) buy in now + 100x (3) don't buy in + worthless (4) don't buy in + 100x (4) is a terrible position to be in, (2) is a good positio…

By that logic you should definitly buy lottery tickets.

Yes you should, if the payoff is high enough or the odds are good enough.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#63

Post by quickthrower2 »

nikanj wrote: post #54
thecolorblue wrote: post #52

I agree with other points here, but I also think nobody wants to wake up 2 years from now and be the only company that was not investing in machine learning. It could turn into nothing, or it could be 100x the time and money put in now. So, of the 4 outcomes: (1) buy in now + worthless (2) buy in now + 100x (3) don't buy in + worthless (4) don't buy in + 100x (4) is a terrible position to be in, (2) is a good positio…

Sounds like a new version of Pascal’s Wager

John Carmack said exactly that in the FB post

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#64

Post by emmamm05 »

peterwoerner wrote: post #12

Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…

Strongly agreed. In my daily job, the exciting, shiny machine learning work is superfluous compared to the strong keyboard smashing of having to parse millions of rows of excel sheets with incomprehensible organization and column naming.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#65

Post by joddystreet »

Software products can be layered into 3 parts -

- data collection (frontend, APIs) - data storage (database, backend) - data visualization (dashboards, analytics, reports)

To bootstrap a startup, the primary people you need would be - a frontend person, a backend person, and a product person. In the beginning, you would be dreaming about the possibility of using ML but would not invest in hiring a data scientist at that stage. The second stage would be to hand over the reins to operations people and let them optimize the internal and external processes and get ready for the launch (growth). These phases can take-up any amount of time, from 2 to 5 years. Finally you handover the reins over to the salespeople and go back from product to services mode (especially true for enterprise software). During this stage, you would be hiring a data scientist and a team of data analysts. After, around 6-8 years of bootstrapping a company. Unless your product relies heavily on the ML algorithm, it can and would always wait.

Figuring out where to first use ML is another challenge. Hiring a data scientist and ask them to tell you what to do is a futile effort.

ML is a useful tech and if you have to keep thinking about utilizing this technology to improve your product and processes. It has to be a part of your long term efforts.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#66

Post by warrenronsiek »

Most companies aren't interesting enough to invest in without some kind of secret sauce. Claiming that they are going to use 'AI and ML' is a way of saying that 'yes, we are a company with tons of competition and little competitive advantage, but we will have a secret sauce eventually. We don't know what that is yet, so we are using jargon as a placeholder.'

Call me a cynic, but I think the talk of building a data moat is mostly nonsense. Of the companies that make these claims, how many of them are actually hiring expensive data engineers to build the moat? If don't have a team working on it, then its a ruse.

Unless the startup is founded by people with deep experience in ML and have been working on using it extensively from day 0, it is unlikely that they will be able to deliver on this vision. They won't be collecting the data correctly if they are doing it at all. If they are collecting it correctly (they aren't), they won't be able to get it to a useable form. If they get that far (they won't), they then need to build out the ML Ops to deliver their models. Now, finally, they can `from tensorflow import *`. Engaging this process post hoc takes YEARS.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#67

Post by littlestymaar »

nostrademons wrote: post #45
papln wrote: post #27

Earlier quoted context omitted.

> data ownership and pipelines not ML. By your argument, the money is in data hoarding and brokering, and renting that data (with DRM) to ML outfits, not doing ML. Anyway, cleaning dirty data isn't execeptionally hard, it's just boring work; getting the raw data is the hard part.

This is an interesting observation that deserves to be addressed. I think in practice, what happens is that there's no easy transactional boundary that can keep the data ownership and ML in separate firms. The theory of the firm [1] states that firms arise when transaction costs are less than the economic inefficiencies of centralized resource allocation. There are some pretty heavy transaction costs between doing da…

> DRM for datasets isn't really a thing, since to explore, visualize, and train on them, you need access to the raw data, and then instead of your machine-learning function you can just pass the identity function to get the raw data.

Functional Encryption[1] is aiming at this exact problem. It's still really early stage, and it currently has a lot of caveats (the current algorithms are really expensive and only able to work with linear function AFAIK), but the field is moving rapidly, and we don't know how far it will get. There's also Fully Homomorphic Encryption [2], which is way cheaper and more versatile, but in that case, only the owner of the data can get the result of the calculation.

None of this is currently used at scale, but their is a lot of research on this field and some PoC are being built at Microsoft and IBM IIRC. There's also Cosmian[3], a French startup working on this topic. (Full disclosure, I know the founders)

[1]: https://en.m.wikipedia.org/wiki/Functional_encryption

[2]: https://en.m.wikipedia.org/wiki/Homomorphic_encryption#Fully...

[3]: https://cosmian.com/

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#69

Post by dawg- »

emmanuel_1234 wrote: post #60
geebee wrote: post #38

Earlier quoted context omitted.

I agree with you, though I actually personally don't find pre-processing data tedious and boring. I kind of like knitting it all together. On a side note... data pre-processing is often viewed a side job that needs to get done before the real work can begin. I don't think I've ever been able to prepare a data pipeline without making decisions about the data that will impact the outcome. For example How do you deal wi…

As a corollary, "Data Scientists" who can't program their way out of a paper bag (e.g.: write simple SQL, or a scraper in Python) is near useless, and a stress on their peer who can. I'd much rather hire a good programmer with some statistical knowledge than the other way around.

Either way, you are gonna be paying someone to fill in gaps in their knowledge. Anyone with a laptop and an internet connection can learn to "program their way out of a paper bag" in a weekend.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#70

Post by peterwoerner »

H8crilA wrote: post #48
peterwoerner wrote: post #12

Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…

You should've bought the processed 10q and 10k data ranging many years for all US stocks. I bet you can get that for under $1000. This is the most basic securities data that exists in finance, perhaps you can even get it for free from some Yahoo or other retail data source.

Yes, but I was in graduate school making $1500/mo and was doing it mostly for fun as a class project. I was also very naive about how much effort it would take to process that data. Both google and yahoo have current data which is easy to get but not historical. Google didn't really help me find what I was looking for.
Post reply on HN