Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to deliver on Machine Learning projects

blog.insightdatascience.com

Post reply on HN
Print view View original

11–20 of 41 posts

Re: How to deliver on Machine Learning projects

#11

Post by probably_wrong »

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

Maybe it's an instance of "when all you have is a hammer...", because I'm learning about it right now, but you could look into transfer learning - you train a ML model in a similar, easier task, and then you tweak it with your data.

That said, there's a good chance that your current algorithm is all you will ever need - many times a ML project is too much, and you already have good results.

Re: How to deliver on Machine Learning projects

#12

Post by superflyguy »

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

"The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!!"

Designing the perfect viola using machine learning doesn't sound like it's something for beginners.

Re: How to deliver on Machine Learning projects

#13

Post by e_ameisen »

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

Co-author here. This is a surprisingly common situation. In fact starting with the simplest algo is usually the best way to prove the validity of your approach, and gather initial data to build a more complex model later.

In addition, trying for the feature to “work perfectly” from the get go, even with lots of data usually is quite hard.

Re: How to deliver on Machine Learning projects

#14

Post by minimaxir »

probably_wrong wrote: post #11
vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

Maybe it's an instance of "when all you have is a hammer...", because I'm learning about it right now, but you could look into transfer learning - you train a ML model in a similar, easier task, and then you tweak it with your data. That said, there's a good chance that your current algorithm is all you will ever need - many times a ML project is too much, and you already have good results.

Transfer learning only works if the original model is in the same domain (e.g. ImageNet for images, GloVe for text). A bespoke problem likely won't have a widely-available original model.

Re: How to deliver on Machine Learning projects

#15

Post by chippy »

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

I've been talking to academic neural net / ML experts in computer vision and OCR / NLP and the thing they try to stress is that for almost all cases an algorithmic approach works better.

Re: How to deliver on Machine Learning projects

#16

Post by e_ameisen »

b_tterc_p wrote: post #2

this doesn’t seem to offers any novel perspectives. I read it as intended for self marketing.

Co-author here. This post came out of a discussion with Adam, where we both realized that the advice we were giving to ML teams and ML Engineers to guide them to better results were very often process centric rather than model centric.

Many resources exist online about how to get a model to converge, and that’s not usually what makes or break a project.

Data acquisition, augmentation, model selection, and iterative exploration however seem quite rarely discussed compared to how important we have seen them be. This is our attempt at sharing this outside of our usual circles.

Re: How to deliver on Machine Learning projects

#17

Post by bonniemuffin »

vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

That seems fine to me. It's a good practice to start with hard-coded business rules instead of any kind of model, just to test the waters, collect some data, and see if a new feature even makes sense, before diving into building even the simplest model.

Re: How to deliver on Machine Learning projects

#18

Post by tedivm »

hellogoodbyeeee wrote: post #8
vjsc wrote: post #5

So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple. Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out t…

They gave you two weeks to become a data scientist and implement a working solution? That's nuts. I'm still pretty early career, but I have done data science work for about four years now and I wouldve quoted at least two months to figure out data, clean it, feature engineer, run models, compare results, and then deliver the best performing solution.

And they didn't even have data!

Re: How to deliver on Machine Learning projects

#19

Post by fromthestart »

Machine Learning is much more nuanced than people seem to understand. You can't just throw data at a net and expect results-this field requires a heavy degree of intuition, and engineers must be prepared for nets to pick up on patterns not obvious to humans, which can lead to unintuitive results.

Neural nets are basically black box heuristics, with unpredictable edge cases. Much like human reasoning, I'd warrant!

Post reply on HN