What it takes to build great machine learning products
radar.oreilly.com
What it takes to build great machine learning products
1–10 of 49 posts
Re: What it takes to build great machine learning products
#2It's a very exciting time. I'm incredibly excited to see what goes on here. I previously explored an online education start-up idea and I'm really looking forward to seeing Ng and Koller change the world. I'm also very exciting to see machine learning on the radar. For me one of the biggest challenges is often making AI intuitive. As machine learning becomes more mainstream it will be on people's design radar and that will make it less hard to turn great algorithms into great products.
Re: What it takes to build great machine learning products
#3I think this is pretty accurate. Here is an example from my own thesis research: I'm using machine learning to tune an (underwater) communication link, i.e. decide what modulation / error coding algorithms/parameters will yield good data rates in a dynamic channel.
At first i tried using an off-the-self classifier to figure out which parameters will work well. That failed because by the time i had sampled a decent proportion of the possible parameter values, the channel would change (the number of possible combinations is of the order of a few millions).
Turned out that the real problem is not learning the performance of the available parameters, rather it lies in "learning how to learn": i.e. my ML system needs to adaptively search the space, by responding to the history of previous explorations and their outcomes. This kind of exploration would be effective only with an understanding of how the underlying modulation/coding algorithms work and interact with each other.
Re: What it takes to build great machine learning products
#4It's a very exciting time. I'm incredibly excited to see what goes on here. I previously explored an online education start-up idea and I'm really looking forward to seeing Ng and Koller change the world. I'm also very exciting to see machine learning on the radar. For me one of the biggest challenges is often making AI intuitive. As machine learning becomes more mainstream it will be on people's design radar and tha…
Partially, although in my experience over the past 4 years doing this stuff 1 hour cleaning the input data gets you thrice the output of 1 hour tuning the algos. Some algorithms are more sensitive than others, but in general, garbage in, garbage out.
Re: What it takes to build great machine learning products
#5Very nice article Aria. You quickly mention Pegasos as a scalable alternative to SMO. I agree that this works well for linear models. But despite the claim that Pegasos can be trivially adapted to kernel models I have never seen any implementation of a kernel Pegasos and I don't understand how it's even possible. Have you used Pegasos-style algorithm to fit non linear models?
On the other hand there exist alternatives such as LaSVM that can effectively scale linearly to large datasets (but the optimizer works in dual representation as with SMO and not like Pegasos).
Re: What it takes to build great machine learning products
#6I agree that the big wins in machine learning/(weak)AI are probably going to come more from figuring out how to better apply existing models and algorithms to real problems rather than from improving the performance of the algorithms themselves.
That said one shouldn't underestimate the amount of commonality between problems that to some people may appear unrelated. For example this post talks about the gains in machine translation performance from including larger contexts. The same principle applies to many other sequence learning problems. For example you have a very similar issue with handwriting recognition where it is often not possible (even for a human) to determine the correct letter classification for a given handwritten character without seeing it within the context of the word.
Re: What it takes to build great machine learning products
#7I think this is pretty accurate. Here is an example from my own thesis research: I'm using machine learning to tune an (underwater) communication link, i.e. decide what modulation / error coding algorithms/parameters will yield good data rates in a dynamic channel. At first i tried using an off-the-self classifier to figure out which parameters will work well. That failed because by the time i had sampled a decent pr…
Indeed. From our own experience: we use pretty much off-the-shelf maximum entropy parameter estimators for parse disambiguation and fluency ranking. In the past ~10 years most of the gain has come from smart feature engineering by using linguistic insights, analyzing common classes of classification errors, etc. Beyond l1 or l2 regularization, the use of (even) more sophisticated machine learning algorithms/techniques have not yet given much, if any, improvement for these tasks in our system.
What did help in understanding models is the application of newer feature selection techniques that give a ranked list of features, such as grafting.
Re: What it takes to build great machine learning products
#8Very nice article Aria. You quickly mention Pegasos as a scalable alternative to SMO. I agree that this works well for linear models. But despite the claim that Pegasos can be trivially adapted to kernel models I have never seen any implementation of a kernel Pegasos and I don't understand how it's even possible. Have you used Pegasos-style algorithm to fit non linear models? On the other hand there exist alternative…
So...are you saying you need the dual formulation in order to allow a kernel model?
Re: What it takes to build great machine learning products
#9My for-profit company (Brighter Planet) often gets product ideas from our data scientists; it's exactly what Dr. Haghighi is talking about.
For example: trying to model environmental impact of Bill Gates's 66,000 sq ft house during a hackathon -> discovery that we need fuzzy set analysis (https://github.com/seamusabshere/fuzzy_infer) -> new, marketable capabilities in our hotel modelling product (https://github.com/brighterplanet/lodging/blob/master/lib/lo...).
Re: What it takes to build great machine learning products
#10The article doesn't mention two important things (and instead focuses on being clever - the opposite of what machine learning stands for). First, the deep learning algorithms that automatically create features. Second, the importance of gathering lots of data, or generating it.
If you have to be really clever with feature engineering, then what's the point of even calling yourself a machine learning person.