Live data from Hacker News

Machine Learning for Developers

xyclade.github.io

21–30 of 99 posts

Re: Machine Learning for Developers

#21

Is there a decent introduction to scala as a language with real world ML? I've come across various ML primers that go into detail on PCA, Linear regression, etc. But not any that show real world ML usage i.e. if person listen to music of type X they'll also like Y. Face detection, etc.

You also don't see things about feature engineering, how to know what to do next to improve things, etc. ML is an art, and nobody covers this much :(

Re: Machine Learning for Developers

#22
post #21

Is there a decent introduction to scala as a language with real world ML? I've come across various ML primers that go into detail on PCA, Linear regression, etc. But not any that show real world ML usage i.e. if person listen to music of type X they'll also like Y. Face detection, etc.

You also don't see things about feature engineering, how to know what to do next to improve things, etc. ML is an art, and nobody covers this much :(

exactly my thoughts. Its one thing to "know" about curse of dimensionality but one doesn't get to knot things unless they are applied to solve problems.

Re: Machine Learning for Developers

#23
post #3

Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.

Java and Scala? Who uses that in ML? Python has long been the best language for ML You're kidding, right? Java has been extremely popular for ML for a long time. Not to cast any shade on Python, but I'd say Java and Python are roughly equivalent in this regard. Both have good libraries for various ML tasks and both are very popular in the domain. For reference, a quick search on mloss.org finds 84 projects identified…

I feel that much like programming in the corporate world, Python is often used to teach ML while Java is more often used to implement it.

Re: Machine Learning for Developers

#24
post #15
post #14

Earlier quoted context omitted.

[deleted]

>Quick job search shows that there are many companies out there that do ML in Scala, e.g. Sharethrough, Teralytics, LeadIQ. What do you use to do a search that specific?

Sounds like he used LinkedIn to me.

Re: Machine Learning for Developers

#25
Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

Re: Machine Learning for Developers

#26

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

Is that any different from a bunch of developers plugging in magic numbers into a formula that they made up, which (to a first approximation) is roughly what happens now?

Realistically, the outcome will be the same as it is now: those firms whose models don't reflect reality will blow up, those whose do will get bigger, a few will get too big to fail off some very confidently-expressed models and make a lot of people mad at them, and eventually the market will straighten out who's lying and who's not. Won't be painless, but then, capitalism never is.

Re: Machine Learning for Developers

#27

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

No because their businesses won't work if they don't understand that stuff. It took us a while to learn it in the anti spam world but not that long in the big scheme of things.

Re: Machine Learning for Developers

#28
post #13
post #3

Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.

Python isn't used for much ML in the field from my experience. It is heavily used for teaching and learning about ML - but for actual production ML, I've seen mostly compiled languages. The main reason is that ML is highly parallelizable and Python isn't terribly good at that. Plus you need to crunch large datasets and speed becomes important. So, respectfully, lots of people use languages other than Python for ML, a…

I've seen Python (and R) used all the time for exploratory ML. Do all of your feature extraction, feature selection, parameter tweaking, and backtesting in Python, and then once you have a model that works reasonably well, port the feature extraction for only the features that actually work well over to a compiled language like Java or C++, train your models on lots of data, and do your actual classification in the compiled language.

Most ML is an iterative process, and the final model that's used in production is just the tip of the iceberg of the development work that went on. Python works as well for exploratory programming there as it does for any other domain.

Re: Machine Learning for Developers

#29
post #13
post #3

Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.

Python isn't used for much ML in the field from my experience. It is heavily used for teaching and learning about ML - but for actual production ML, I've seen mostly compiled languages. The main reason is that ML is highly parallelizable and Python isn't terribly good at that. Plus you need to crunch large datasets and speed becomes important. So, respectfully, lots of people use languages other than Python for ML, a…

I've seen Python used heavily in ML where much of the work is done on the GPU anyways. There are plenty of libraries for that, and Python's slow nature isn't as relevant (though it still becomes relevant when CPU processing is the bottleneck!).

Re: Machine Learning for Developers

#30

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

There are plenty of people who don't know their algorithms and data structures writing prodduction code for large companies.
Post reply on HN