Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What it takes to build great machine learning products

radar.oreilly.com

Post reply on HN
Print view View original

31–40 of 49 posts

Re: What it takes to build great machine learning products

#31

Post by TimPC »

3pt14159 wrote: post #4
TimPC wrote: post #2

It's a very exciting time. I'm incredibly excited to see what goes on here. I previously explored an online education start-up idea and I'm really looking forward to seeing Ng and Koller change the world. I'm also very exciting to see machine learning on the radar. For me one of the biggest challenges is often making AI intuitive. As machine learning becomes more mainstream it will be on people's design radar and tha…

Partially, although in my experience over the past 4 years doing this stuff 1 hour cleaning the input data gets you thrice the output of 1 hour tuning the algos. Some algorithms are more sensitive than others, but in general, garbage in, garbage out.

I think in many cases you're correct. My point wasn't about the performance of the AI algorithms themselves though. In my experience most of the problems where I want to use AI the algorithm itself performs adequately. Getting the interaction with the algorithm sensible for a non-technical user is hard. If AI becomes prevalent enough that UI/UX people start thinking about it, I suspect it will be much easier to solve that problem, which to me is the bigger business problem with AI.

Re: What it takes to build great machine learning products

#32

Post by Dn_Ab »

suneilp wrote: post #22
chaostheory wrote: post #16

The article is light on details. imo there are two major things your team needs: 1) Programmers that have the needed math skills, or mathematicians with the needed coding skills 2) A distributed ML framework Solving problem one is not easy but it's straightforward. Solving problem two is harder. While there are a lot of open source machine learning projects, almost all of them seem to have a focus of being used by a…

What kind of math skills? What would a programmer need to learn in order to work on ML stuff?

To work with ML stuff you don't need much. You can just download packages and get experience with how to best pick models, choose features and tweak (hyper)parameters. If you want to work on or understand it then you will need math.

You need a decent understanding of calculus (mid 1800s level, mulitvariate calculus), a more decent understanding of Linear Algebra (1950s ), information theory (1960s), and probability and statistics. With the last having shifted the most from the past due to more recent respect for bayesian methods. Note the years in parenthesis is not to say that nothing new has been used from those areas, more like if you pick up a book on that topic from that year you would be pretty well covered for the purposes of ML.

Worth having a vague idea of are stuff like PAC learning, topology and computational complexity stuff like Valiant's work on evolvability. If you are doing stuff related to genetic programming then category and type theory have riches to be plundered.

Or if you want to be more hardcore and are looking at very higher dimensional data and reductions on them you might look at algebraic geometry (in particular algebraic varieties) and group theory. So basically the answer to your question is as little or as much math as you want and or depending on the problem and your interests in trying different approaches than the typical toolkits of linear algebra and statistics.

Re: What it takes to build great machine learning products

#34

Post by ma2rten »

brendano wrote: post #18
ma2rten wrote: post #14

Right now NLP is mostly limited to niche applications, like e.g. sentiment analysis and clever products build around it. I actually think the reason is that both natural language processing and machine learning are still in their early days. Imagine all the applications for consumer products if algorithms would be really able to actually understand language (as far as you can understand something if you are a compute…

Careful with the Collobert ICML-2008 paper. It has a very negative reputation among NLP researchers who actually know the area, just for its setup/evaluation. If you're interested in the methods (which I think are interesting), that group's later work is much improved.

Thanks I will look into it.

Re: What it takes to build great machine learning products

#35

Post by [deleted] »

robrenaud wrote: post #28
irahul wrote: post #24

Earlier quoted context omitted.

> On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. I find data cleansing(if you are including feature selection) hard, and I consider it a refinement. If I am working on a classification problem, I start with naive bayes with a trivial feature generator(if words are feature, split on whitespace…

Naive Bayes is a linear classifier, and it makes much stronger assumptions than other linear classifiers.

[deleted]

Re: What it takes to build great machine learning products

#36

Post by psb217 »

ogrisel wrote: post #5

Very nice article Aria. You quickly mention Pegasos as a scalable alternative to SMO. I agree that this works well for linear models. But despite the claim that Pegasos can be trivially adapted to kernel models I have never seen any implementation of a kernel Pegasos and I don't understand how it's even possible. Have you used Pegasos-style algorithm to fit non linear models? On the other hand there exist alternative…

You may want to look at the paper: "P-packSVM: Parallel Primal grAdient desCent Kernel SVM" from ICDM 2009. It presents an extension of Pegasos to non-linear kernels. Evaluating the pairwise kernels and continuously updating the estimate of the norm of the implicit weight vector w seem to be the main hurdles to achieving the performance gains seen with linear kernels.

The key takeaway from the paper (for me) was that the computation time on a single processor was not significantly better than that of the standard implementation provided by SVM-Light. However, with a variety of tricks permitted by the use of an SGD/Pegasos-like method, the authors were able to get significant speedup when using a compute cluster, allowing a good reduction in computation times (e.g. ~200x reduction on 512 processors).

Re: What it takes to build great machine learning products

#37

Post by irahul »

robrenaud wrote: post #28
irahul wrote: post #24

Earlier quoted context omitted.

> On the other hand, people who know a bit about ML but understand the domain better start by applying intuition to data cleansing and then follow up with simpler algorithms. I find data cleansing(if you are including feature selection) hard, and I consider it a refinement. If I am working on a classification problem, I start with naive bayes with a trivial feature generator(if words are feature, split on whitespace…

Naive Bayes is a linear classifier, and it makes much stronger assumptions than other linear classifiers.

My bad.

Regarding stronger assumptions, is there anything other than independence(thus the name naive) that it assumes?

Re: What it takes to build great machine learning products

#38

Post by nkh »

Dn_Ab wrote: post #32
suneilp wrote: post #22

Earlier quoted context omitted.

What kind of math skills? What would a programmer need to learn in order to work on ML stuff?

To work with ML stuff you don't need much. You can just download packages and get experience with how to best pick models, choose features and tweak (hyper)parameters. If you want to work on or understand it then you will need math. You need a decent understanding of calculus (mid 1800s level, mulitvariate calculus), a more decent understanding of Linear Algebra (1950s ), information theory (1960s), and probability a…

* If you are doing stuff related to genetic programming then category and type theory have riches to be plundered.*

Could you expand this a bit as I don't understand the meaning. Are you saying that if the problem you are working on can be solved with genetic algorithms, then you could blow it away with category and type theory?

I don't have a vested interest in either, I am just curious. Thanks.

Re: What it takes to build great machine learning products

#39

Post by grinalds »

marshallp wrote: post #10

The article doesn't mention two important things (and instead focuses on being clever - the opposite of what machine learning stands for). First, the deep learning algorithms that automatically create features. Second, the importance of gathering lots of data, or generating it. If you have to be really clever with feature engineering, then what's the point of even calling yourself a machine learning person.

Deep learning is an interesting approach - although the features that DL algorithms decide are most important are not always intuitive or weighted properly in context. Partial-feature engineering is sometimes the only way to effectively deal with biases, especially in higher-dimensional space where the DL features can be very opaque.

Re: What it takes to build great machine learning products

#40

Post by tensor »

irahul wrote: post #37
robrenaud wrote: post #28

Earlier quoted context omitted.

Naive Bayes is a linear classifier, and it makes much stronger assumptions than other linear classifiers.

My bad. Regarding stronger assumptions, is there anything other than independence(thus the name naive) that it assumes?

That's the main assumption that people care about. In contrast, logistic regression (aka maximum entropy) does not make this assumption. As a go-to first classifier I would suggest multi-class logistic regression with regularization.
Post reply on HN