Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What it takes to build great machine learning products

radar.oreilly.com

Post reply on HN
Print view View original

21–30 of 49 posts

Re: What it takes to build great machine learning products

#21

Post by mturmon »

gfodor wrote: post #12

A great and insightful article. A common theme I've seen in practice is folks who have a deep understanding of ML often run straight to applying the most sophisticated algorithms possible on raw data. On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. Without fail the latter group ends up with bet…

Agreed. Another ingredient is sustained engagement with the problem, so that your algorithm works not just for a pre-selected demo, but actually provides noticeable performance gains for real data.

Re: What it takes to build great machine learning products

#22

Post by suneilp »

chaostheory wrote: post #16

The article is light on details. imo there are two major things your team needs: 1) Programmers that have the needed math skills, or mathematicians with the needed coding skills 2) A distributed ML framework Solving problem one is not easy but it's straightforward. Solving problem two is harder. While there are a lot of open source machine learning projects, almost all of them seem to have a focus of being used by a…

What kind of math skills? What would a programmer need to learn in order to work on ML stuff?

Re: What it takes to build great machine learning products

#23

Post by srconstantin »

ogrisel wrote: post #5

Very nice article Aria. You quickly mention Pegasos as a scalable alternative to SMO. I agree that this works well for linear models. But despite the claim that Pegasos can be trivially adapted to kernel models I have never seen any implementation of a kernel Pegasos and I don't understand how it's even possible. Have you used Pegasos-style algorithm to fit non linear models? On the other hand there exist alternative…

Link to LaSVM paper: jmlr.csail.mit.edu/papers/volume6/bordes05a/bordes05a.pdf Also a good overview of SVM techniques in general.

Re: What it takes to build great machine learning products

#24

Post by irahul »

gfodor wrote: post #12

A great and insightful article. A common theme I've seen in practice is folks who have a deep understanding of ML often run straight to applying the most sophisticated algorithms possible on raw data. On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. Without fail the latter group ends up with bet…

> On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms.

I find data cleansing(if you are including feature selection) hard, and I consider it a refinement. If I am working on a classification problem, I start with naive bayes with a trivial feature generator(if words are feature, split on whitespace and discard some symbols), train it, and cross validate. Depending on the results of the cross validation on differently sized data-sets(say 100 tweets, 200, 500, 1000, 2000, 5000) I decide if I refine bayes further or I need to pick another algorithm.

I avoid SVM because I have a hard time figuring out the kernel and relation between data. I mostly don't use linear classifiers because the relation is very rarely linear.

Generally if the features are pseudo-independent(naive bayes assumes independent events but it might work fine even if the events aren't independent), naive bayes does the job. If not, it's time to refine the feature generator and selector.

Re: What it takes to build great machine learning products

#25

Post by salimmadjd »

suneilp wrote: post #22
chaostheory wrote: post #16

The article is light on details. imo there are two major things your team needs: 1) Programmers that have the needed math skills, or mathematicians with the needed coding skills 2) A distributed ML framework Solving problem one is not easy but it's straightforward. Solving problem two is harder. While there are a lot of open source machine learning projects, almost all of them seem to have a focus of being used by a…

What kind of math skills? What would a programmer need to learn in order to work on ML stuff?

Aside from algebra to do log or exp division addition and multiplications, you have to be versed in statistics. Most ml problems are solved on statistical bases. Although many of the algorithms have been solved, you still need to grasp the statistics behind it which is a bit more involved than calculating the odds of a die

Re: What it takes to build great machine learning products

#26

Post by salimmadjd »

I have enjoyed the author's other posts via his prismatic blog here. It's one of the most interestings blogs to follow with only a few posts so far. However, this article falls a bit short. It feels rushed out, which is understandable.

I think it would have been better if this was just the first part of a multi-article write up on ML. With this one being an intro and follow-ups on specific approaches.

Re: What it takes to build great machine learning products

#27

Post by chaostheory »

salimmadjd wrote: post #25
suneilp wrote: post #22

Earlier quoted context omitted.

What kind of math skills? What would a programmer need to learn in order to work on ML stuff?

Aside from algebra to do log or exp division addition and multiplications, you have to be versed in statistics. Most ml problems are solved on statistical bases. Although many of the algorithms have been solved, you still need to grasp the statistics behind it which is a bit more involved than calculating the odds of a die

Yes, many people that I know working on ML do not remember statistics well enough. Some keep their ignorance while relying on mathematicians on-staff (who can't really code), while others either start buying college textbooks or go to night classes. There are too many people who don't understand the algorithms being used.

Re: What it takes to build great machine learning products

#28

Post by robrenaud »

irahul wrote: post #24
gfodor wrote: post #12

A great and insightful article. A common theme I've seen in practice is folks who have a deep understanding of ML often run straight to applying the most sophisticated algorithms possible on raw data. On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. Without fail the latter group ends up with bet…

> On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. I find data cleansing(if you are including feature selection) hard, and I consider it a refinement. If I am working on a classification problem, I start with naive bayes with a trivial feature generator(if words are feature, split on whitespace…

Naive Bayes is a linear classifier, and it makes much stronger assumptions than other linear classifiers.

Re: What it takes to build great machine learning products

#29

Post by tel »

gfodor wrote: post #12

A great and insightful article. A common theme I've seen in practice is folks who have a deep understanding of ML often run straight to applying the most sophisticated algorithms possible on raw data. On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. Without fail the latter group ends up with bet…

I think there are essentially two "deep" understandings of ML prevalent today. The first is more common: the ability to do the calculus, algebra, and probability derivations required to design complex ML algorithms combined with the CS knowledge to find/design a good algorithm and the software design skill to actually implement it on real, "big" data.

No doubt this is a difficult position to master and those who perform well are able to tackle lots of mathematical and computational challenges. They also are model builders who (have tendency to) relentlessly seek complex models in order to solve complex problems.

The other, rarer side is the learning theorist who may or may not understand the model building, algorithmic, and computational tools but understands well the theories which allow us to have reasonable expectations that the tools of the first group will work at all. These guys have a funny story in that they were the old statisticians who got a major egg-on-the-face after proclaiming that essentially all of ML was impossible. Turns out the first group managed to redefine the problem slightly and make major headway (and money).

---

The thing I want to bring to light however is that the second group knows the math that bounds the capacities of ML algorithms. This isn't easy. It's one thing to say you recognize that the curse of dimensionality exists, but it's another to have felt it's mathematical curves and to build an intuition for what forces are sufficient to cause disruption.

The more experience you have with the learning maths, the more likely you are, I feel, to apply very simple algorithms, to be scared of "little x's" (real data) enough to treat it with great care, and to attempt to explore the problem space with confidence for what steps will lead you to folly.

---

It's a fine line between the two, though. Stray too far to the first group and you'll spend a month building an algorithm that does a millionth of a percentage point better than Fisher's LDA. Spend too much time in the second camp and you'll confidently state that no algorithm exists that does better than a millionth of a percentage point over Fisher LDA... and then lose purely by never trying.

Post reply on HN