Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do so many startups claim machine learning is their long game?

news.ycombinator.com

Post reply on HN
Print view View original

71–78 of 78 posts

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#71

Post by geebee »

dawg- wrote: post #69
emmanuel_1234 wrote: post #60

Earlier quoted context omitted.

As a corollary, "Data Scientists" who can't program their way out of a paper bag (e.g.: write simple SQL, or a scraper in Python) is near useless, and a stress on their peer who can. I'd much rather hire a good programmer with some statistical knowledge than the other way around.

Either way, you are gonna be paying someone to fill in gaps in their knowledge. Anyone with a laptop and an internet connection can learn to "program their way out of a paper bag" in a weekend.

I kinda disagree, though I suppose it depends on the paper bag.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#72

Post by JohnFen »

tetha wrote: post #24

Ugh. Maybe I'm in a different world by now, but I dislike such statements on multiple levels. > This typically comes with assertions such as "data is the new oil" and "once we have our dataset and models the Big Tech shops will have no choice but to acquire us". Maybe it's me, but I dislike the attitude to work to be acquired. Interestingly, this is a rift I see quite a bit if I interview more development oriented gu…

> I dislike the attitude to work to be acquired.

This is my preferred method of doing business. I start a company with the intention of selling it to someone else in the end. I do this because it fits my personality the best -- once I have solved a technical problem, I lose interest in it and want to move on to the next thing.

However, there's a right way and a wrong way to do business with this sort of goal, and your comment here puts the finger on the difference:

> Build something to last and generate value for a long time.

This is an essential part of what I need to do in order to sell. It's not time to sell my company until the company is already profitable and set to work over the long term. That's where the real value proposition is for buyers -- I'm not selling a technology or piece of software, I'm selling an established business.

In my opinion, people who start businesses with the intent to dump them at the soonest opportunity are really just doing the same thing as people who make money flipping stocks. There's nothing wrong with that (it's just not for me), but it's a completely different sort of thing. It's money-spinning without the intention to build anything of lasting value.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#73

Post by dsaavy »

axegon_ wrote: post #15
peterwoerner wrote: post #12

Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…

So much this!!! In all fairness, it doesn't matter what you pick up, you'll spend north of 80% of your time preparing and pre-processing data, which is almost always catastrophically tedious and boring. Annoyingly that also applies to publicly available datasets - pre-processing is still most of the work. For instance, the tensorflow team invested a lot of time and effort into tf.data for that reason but imo it doesn…

Yep, I actually started a company that cleans data reliably because of this. It’s usually a huge waste of resources for a highly paid person to clean data (like ML specialists) and most data cleaning companies just outsource to overseas resources that aren’t reliable or knowledgeable beyond an Excel skill set.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#74

Post by variaga »

solidasparagus wrote: post #47
variaga wrote: post #29

Earlier quoted context omitted.

Recurring expenses (i.e. the kind that will generate enough data to adequately train a ML system) are already handled pretty well by most businesses through traditional (non-ML) methods. Operations Research has been a thing for 60+ years. Businesses that don't handle their recurring expenses well now are likely to have organizational issues (e.g. senior management that ignores the advice of their reports on how to do…

I completely disagree with the assumption that because something has been done for 60+ years it won't be improved by new, extremely relevant technologies. Amazon is very effective at both OR and organizational dynamics, but they found massive savings by using ML to predict demand and thus inventory/costs (Research is here - https://arxiv.org/pdf/1711.11053.pdf ).

Unless I missed it, the referenced paper doesn't quantify what savings Amazon achieved, or even if Amazon actually used the NN described. It does not support the statement "found massive savings".

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#75

Post by solidasparagus »

variaga wrote: post #74
solidasparagus wrote: post #47

Earlier quoted context omitted.

I completely disagree with the assumption that because something has been done for 60+ years it won't be improved by new, extremely relevant technologies. Amazon is very effective at both OR and organizational dynamics, but they found massive savings by using ML to predict demand and thus inventory/costs (Research is here - https://arxiv.org/pdf/1711.11053.pdf ).

Unless I missed it, the referenced paper doesn't quantify what savings Amazon achieved, or even if Amazon actually used the NN described. It does not support the statement "found massive savings".

No, that was from hearing them present their work.

But I do think the GEFCom2014 Electricity Forecasting benchmark is pretty clear proof that ML solutions such as these can improve expenditure decision making compared to established techniques.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#76

Post by sgt101 »

tlb wrote: post #4

There are surely some startups for which this is bullshit. But the good version of it is: - take some valuable task that's never been successfully automated before - do it manually (and expensively) for a while to acquire data - build an automated system with some combination of regular software and ML models trained on the data - now you can do a valuable task for free - scale up and profit The risk is that it's har…

There's another risk - which is rare events. Basically most of the reason why these workflows are not automated already is that there are a host of "once a week per operative" corner cases that don't show in the data for a long time, and when they do show they don't show as anything but noise. But this is where the human intelligence is spent and this is why the jobs are so hard to automate.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#77

Post by omar_a1 »

emmanuel_1234 wrote: post #60
geebee wrote: post #38

Earlier quoted context omitted.

I agree with you, though I actually personally don't find pre-processing data tedious and boring. I kind of like knitting it all together. On a side note... data pre-processing is often viewed a side job that needs to get done before the real work can begin. I don't think I've ever been able to prepare a data pipeline without making decisions about the data that will impact the outcome. For example How do you deal wi…

As a corollary, "Data Scientists" who can't program their way out of a paper bag (e.g.: write simple SQL, or a scraper in Python) is near useless, and a stress on their peer who can. I'd much rather hire a good programmer with some statistical knowledge than the other way around.

I've seen so much bad math and misunderstood statistics hard coded into widely-implemented software that that's not ideal either.

Ultimately, you need a team with a combination of strengths if your product requires multidisciplinary work. Otherwise you end up with hilariously wrong equations/assumptions in your code base. But, full disclosure, my background is math/science.

Re: Ask HN: Why do so many startups claim machine learning is their long game?

#78

Post by foldingmoney »

peterwoerner wrote: post #12

Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…

>Time to setup python scripts which could parse through the excel spreadsheets, figure out which row corresponded to gross profit margin, and a few other "standard" metrics: ???

If you just want basic financial data there are easier ways, e.g. the Quandl API (not affiliated).

But the bigger issue is that you can't accurately label your training data. A subsequent stock price move could have been due by something revealed in the 10-Q that the market somehow missed, or it could have been caused by a forest fire in Bolivia, or a competitor getting acquired, or any combination of near infinite external events that couldn't have been predicted from the 10-Q.

Post reply on HN