How to deliver on Machine Learning projects • Hacker News Board

Re: How to deliver on Machine Learning projects

#31

Post by MasterScrat » Fri, Oct 05, 2018, 5:15 PM UTC

superflyguy wrote: post #12

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

"The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!!" Designing the perfect viola using machine learning doesn't sound like it's something for beginners.

Yes, sorry to be "that guy" as well, but it's voila ("voilà" if you want to be pedantic).

"Viola" either refers to a stringed instrument, or means "raped" in the sense "he raped" ("il viola"). So please don't use it as an interjection.

Re: How to deliver on Machine Learning projects

#32

Post by rfeather » Fri, Oct 05, 2018, 6:15 PM UTC

The point about hacking away at the code needs to be couched heavily. It's too easy to conclude you've got negative or positive results when what you really have is a silly little bug. The lack of focus on implementation skills in data (or even "real" science) is frightful. The one take away anyone trained in software engineering could share is that if you aren't very sure if it is working as intended, it's very likely not. Code review is very applicable here when making major pivots, even if unit or other testing is decidedly too time consuming for the train test improve loop.

Edit: typo "of" to "if". Somewhat serendipitous if you think about it.

Re: How to deliver on Machine Learning projects

#33

Post by ellisv » Fri, Oct 05, 2018, 6:17 PM UTC

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

> Unfortunately, it turned out to be a chicken and egg problem. For the feature to work perfectly we needed a large amount of training data to train our models. But without the feature actually deployed, we didn't have any way to collect any training data.

Everyone outside of data science seems really surprised by this and I can't count the number of times someone has asked me to build an algorithm for X but has none of the data to support doing so. It doesn't mean the feature/product can't be built but they often want a supervised learning solution without the cost (and time) of acquiring the ground truth data.

Re: How to deliver on Machine Learning projects

#34

Post by S4M » Fri, Oct 05, 2018, 6:49 PM UTC

MasterScrat wrote: post #31

superflyguy wrote: post #12

Earlier quoted context omitted.

"The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!!" Designing the perfect viola using machine learning doesn't sound like it's something for beginners.

Yes, sorry to be "that guy" as well, but it's voila ("voilà" if you want to be pedantic). "Viola" either refers to a stringed instrument, or means "raped" in the sense "he raped" ("il viola"). So please don't use it as an interjection.

I think everybody understood that "viola" was a typo.

Re: How to deliver on Machine Learning projects

#35

Post by sgt101 » Fri, Oct 05, 2018, 6:55 PM UTC

So we do the loop 50 time and we now have an algorithm that works (97%!) on the test set. We are happy! We run it in production and everything looks good (prbly 92% ish). Everyone is happy! We all get promoted or get new jobs. Then, one day, someone actually looks at what it's doing... and lo. It. does. not. work (~51%) Everyone is sad. Apart from us! Yay!

Seriously - an optimisation loop on a test set? Seriously?

Re: How to deliver on Machine Learning projects

#36

Post by claytonjy » Fri, Oct 05, 2018, 7:45 PM UTC

pedrosorio wrote: post #27

yonkshi wrote: post #21

Earlier quoted context omitted.

I don't think most ML experts would agree with that, a big reason DL became popular are the huge improvements they brought to CV and NLP fields. In many ways, traditional approaches were harder because you need huge amount of domain expertise in CV & NLP, whereas a ML expert can solve simple CV problems with almost no domain knowledge. Now, a lot of the business data, especially time series data, I agree that an algo…

"traditional approaches" in CV & NLP were also ML (a quick reminder that machine learning existed long before the deep learning hype). Not sure what the parent meant by "algorithmic approach" though.

yes, but before the ML step the old approaches relied on expert-crafted features. The breakthroughs in those fields via deep-learning is because people found architectures (CNN/RNNs) that could learn those features much, much, much more efficiently than they could be hand-crafted.

Re: How to deliver on Machine Learning projects

#37

Post by AlexanderNull » Fri, Oct 05, 2018, 8:53 PM UTC

b_tterc_p wrote: post #2

this doesn’t seem to offers any novel perspectives. I read it as intended for self marketing.

I hate quotes but there's a single one I'll ever use because it's not only accurate but incredibly useful: "People need to be reminded more often than they need to be instructed."

Re: How to deliver on Machine Learning projects

#38

Post by JHonaker » Sat, Oct 06, 2018, 12:53 AM UTC

tedivm wrote: post #18

hellogoodbyeeee wrote: post #8

Earlier quoted context omitted.

They gave you two weeks to become a data scientist and implement a working solution? That's nuts. I'm still pretty early career, but I have done data science work for about four years now and I wouldve quoted at least two months to figure out data, clean it, feature engineer, run models, compare results, and then deliver the best performing solution.

And they didn't even have data!

No data is better than 10 years of useless data. I’d much rather be in the position of designing the data collection (experimental design ftw) than trying to fix the problems with an overly complicated modeling project. Buuut, I am a statistician.

In my experience, having someone that knows what they’re doing on the front end of a study design wise can save weeks or months of work on the back end of a study or project.

Re: How to deliver on Machine Learning projects

#39

Post by speby » Sat, Oct 06, 2018, 3:24 AM UTC

hellogoodbyeeee wrote: post #8

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

They gave you two weeks to become a data scientist and implement a working solution? That's nuts. I'm still pretty early career, but I have done data science work for about four years now and I wouldve quoted at least two months to figure out data, clean it, feature engineer, run models, compare results, and then deliver the best performing solution.

> They gave you two weeks to become a data scientist and implement a working solution? That's nuts.

Oh c'mon. Any large company today and the expectation or deadline for practically anything is "asap" or measured in a few weeks at most. Short-term thinking is a major player in publicly traded companies. Because of that, this is what opens the door for startups to play the long-game.

Re: How to deliver on Machine Learning projects

#40

Post by borroka » Sat, Oct 06, 2018, 4:23 AM UTC

reureu wrote: post #30

ende wrote: post #25

Earlier quoted context omitted.

Well, “Data Scientist” has been appropriated by the overflow of PhD’s w/o any actual stats or computational backgrounds and few academia prospects, so I guess you need to create new job titles for thise who are going to do the actual work.

I totally agree, and wasn't arguing that a new title wasn't necessary. And I'm ok with my downvotes for that comment :) It's just funny that "Data Scientist" seemed to be originally branded as the more technical/engineer-y version of a data analyst. Now I get recruiters contacting me for "Data Scientist" positions that entirely revolves around SQL and excel, and nobody in the Bay Area hires "Data Analysts" anymore. A…

I do not see any problem with that. There is a ton of confusion in the tech world regarding labels, who does what, it is needed or not, outside of the core actions that need to be done. The net effect of laying off 50% of tech people from public tech companies might even result in a net positive for the companies. Not for a tech worker like me, so please do not tell them.

Taking advantage as much as possible of hypes and other people's lazyness is fine in my book. It is certainly not my duty from the outside to educate recruiters and business people who make hiring decisions on the field – when I tried, from the inside, to gently point out that what they were thinking did not make any sense, I just put myself in a dangerous spot. I can be a data scientist, deep learning engineer, machine learning engineer, machine learning research scientist, whatever pays more and whoever has the most fun. If using an RNN instead of a more effective and efficient linear regression gives me more money and prestige, I will do it – as an IC you either go with the flow or you are not having a good time. The vast majority of us is not saving lives anyway.