How Google Is Remaking Itself for “Machine Learning First” • Hacker News Board

Re: How Google Is Remaking Itself for “Machine Learning First”

#111

Post by giardini » Thu, Jun 23, 2016, 4:56 PM UTC

randcraw wrote: post #106

tdullien wrote: post #99

Earlier quoted context omitted.

I am a mathematician by trade, and was doing development along with other stuff (reverse engineering and security work, first in my own company, then at Google). So ... 1) I think working knowledge of ML is extremely useful to many developers, and generally under-taught in universities. See the old Joel article which mentions "Google uses Bayesian filtering like MS uses the IF statement" http://www.joelonsoftware.com…

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value. Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations…

I don't know why randcraw is being downvoted here: his/her points are vital clarifications.

Humans have been gathering and analyzing data for thousands of years. We have _not_ waited for Google's latest ML or neural nets to do analyses. Otherwise I'd be carving this post onto a stone for future generations to peruse.

The valuable and understandable AI, the step that will make a difference, isn't in "big data" - it's in figuring out how to do what those humans have been doing all those thousands of years.

Re: How Google Is Remaking Itself for “Machine Learning First”

#112

Post by dgacmu » Thu, Jun 23, 2016, 4:56 PM UTC

randcraw wrote: post #106

tdullien wrote: post #99

Earlier quoted context omitted.

I am a mathematician by trade, and was doing development along with other stuff (reverse engineering and security work, first in my own company, then at Google). So ... 1) I think working knowledge of ML is extremely useful to many developers, and generally under-taught in universities. See the old Joel article which mentions "Google uses Bayesian filtering like MS uses the IF statement" http://www.joelonsoftware.com…

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value. Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations…

Think outside consumer-facing applications. Medicine, biology, geology (oil, gas, and mining), finance, transportation. Tons of data, tons of dollars, and important problems.

Re: How Google Is Remaking Itself for “Machine Learning First”

#113

Post by xenihn » Thu, Jun 23, 2016, 5:17 PM UTC

krosaen wrote: post #104

xenihn wrote: post #9

Anyone happen to have a suggested self-teaching path for Machine Learning? I.e. books and courses. I know that Andrew Ng's course is a great resource, but I know that I'm not ready to start it yet. I'm actually way behind on the mathematical pre-requisites, so recommendations for that would be greatly appreciated as well. I've never taken a statistics course, and never received any formal education for mathematics pa…

I'm taking time off to study ML and keep an ongoing list of curriculum resources, as well as a blog of my day to day, here: http://karlrosaen.com/ml/

Thanks for this! I can see that you and I are somewhat on the same page in terms of mindset, though you're far ahead of me when it comes to both dev experience and math.

Re: How Google Is Remaking Itself for “Machine Learning First”

#114

Post by giardini » Thu, Jun 23, 2016, 5:51 PM UTC

dgacmu wrote: post #112

randcraw wrote: post #106

Earlier quoted context omitted.

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value. Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations…

Think outside consumer-facing applications. Medicine, biology, geology (oil, gas, and mining), finance, transportation. Tons of data, tons of dollars, and important problems.

Most of that data isn't "big data". And most of it has been analyzed thoroughly. Sure, ML will be used to re-analyze it, but with mostly the same results. As randcraw states "Most/all of the value has been extracted."

Only a wild-eyed ML "gold digger" could imagine that there is a vein of gold in those mines. The reality is that, with few exceptions, we'll find more lumps of coal.

Perhaps I should switch from an ML swamp metaphor to an ML mine metaphor? <--Hah! Do that with ML!

Re: How Google Is Remaking Itself for “Machine Learning First”

#115

Post by randcraw » Thu, Jun 23, 2016, 7:25 PM UTC

dgacmu wrote: post #112

randcraw wrote: post #106

Earlier quoted context omitted.

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value. Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations…

Think outside consumer-facing applications. Medicine, biology, geology (oil, gas, and mining), finance, transportation. Tons of data, tons of dollars, and important problems.

I work in a big pharma analyzing image and experimental data. In a prior life I analyzed social cliques from vast numbers of user transactions. In both cases it seems like greater volumes of data should lead to deeper insights. But as it happens, the amount of useful actionable information in that data was surprisingly limited.

Often the available sensors/assays failed to detect reliable info. Or the phenomenon of interest interdepended on too many variables expressed with too great a dynamic range for us to detect reliably or model usefully. (The present lull in genomics R&D illustrates this well, as do automated interpretation of signals like EEG and NMR spectra.) And the signals that we can extract are often uninterpretable or sporadic. Alas, gathering more data won't yield more signal. Given the present limit on sensor resolution, you just get more mixed signals.

The potential of all ML is limited by the depth of the data that are essential for the discrimination of subtler signals. In the domains you mention (medicine, biology, geology, other sciences) I'm convinced we need better sensors more than greater amounts of the same data available now. We need better hypotheses which lead to better ideas of where to look and what to look for. In general, ML can't help with that. Until we better imagine how the mechanism might work, our questions remain too vague.

To wit, I'm afraid that applying ML to most software apps will suffer from the same limited ROI. I suspect that most app and user data is too shallow for mining to add appreciable value, no matter how clever it is.

Re: How Google Is Remaking Itself for “Machine Learning First”

#116

Post by quonn » Thu, Jun 23, 2016, 7:57 PM UTC

argonaut wrote: post #89

visarga wrote: post #79

Earlier quoted context omitted.

There are two major cases: academic, related to algorithm design and industry - related to deployment of already existing algorithms on various data sets. For a CS engineer who wants to be able to use the latest Inception neural net from Google in his pipeline, there is actually almost zero math need. It's like any other API. In goes the image, out comes the label. What she would need to know, as a good utilizer of M…

That isn't "doing machine learning," for the same reason that web developers aren't "operating systems programmers" (even though they use operating systems and need to know some OS concepts).

So, what do you expect your ML engineer does all day? Most of them do in fact spend much time "applying".

Re: How Google Is Remaking Itself for “Machine Learning First”

#117

Post by serge2k » Thu, Jun 23, 2016, 7:58 PM UTC

giardini wrote: post #111

randcraw wrote: post #106

Earlier quoted context omitted.

I don't think your points are invalid, but I think you overvalue the data that's available and relevant to most programming tasks. And without novel data, ML can offer little novel value. Google, Facebook, M$ Research, and perhaps Yahoo are extreme outliers. They have zottabytes of broad unstructured text data, so they mine it. Everybody else has megabytes of narrow structured data, most of it commercial transations…

I don't know why randcraw is being downvoted here: his/her points are vital clarifications. Humans have been gathering and analyzing data for thousands of years. We have _not_ waited for Google's latest ML or neural nets to do analyses. Otherwise I'd be carving this post onto a stone for future generations to peruse. The valuable and understandable AI, the step that will make a difference, isn't in "big data" - it's…

> I don't know why randcraw is being downvoted here

I can't speak for anyone else, but "M$"

Re: How Google Is Remaking Itself for “Machine Learning First”

#118

Post by serge2k » Thu, Jun 23, 2016, 8:00 PM UTC

etangent wrote: post #64

arbre wrote: post #33

I don't believe in "everyone should work on machine learning". I worked on several deep learning models but I don't really like it. It is a very different job than software engineering in my opinion. ML is more about gathering data and tuning the models as opposed to building stuff. I have spent months working on models and barely wrote any code. It is more efficient to have ML experts focus on the modeling and softw…

I don't know. I know some engineers who have spent months going back-and-forth over communication protocols while barely writing any code, yet somehow their job is considered to be quite core to software engineering. I don't really see how fine-tuning communication protocols is fundamentally different from fine-tuning machine learning models. But overall, I agree with your sentiment: different things are different an…

Wouldn't that be more akin to the design of the models?

Re: How Google Is Remaking Itself for “Machine Learning First”

#119

Post by argonaut » Thu, Jun 23, 2016, 11:32 PM UTC

quonn wrote: post #116

argonaut wrote: post #89

Earlier quoted context omitted.

That isn't "doing machine learning," for the same reason that web developers aren't "operating systems programmers" (even though they use operating systems and need to know some OS concepts).

So, what do you expect your ML engineer does all day? Most of them do in fact spend much time "applying".

Developing new or improved models.

Re: How Google Is Remaking Itself for “Machine Learning First”

#120

Post by bhntr3 » Sat, Jun 25, 2016, 12:39 AM UTC

collyw wrote: post #101

bhntr3 wrote: post #45

Earlier quoted context omitted.

I actually think the hardest part about ML is the lingo. It's very alienating that even simple concepts seem to have their own lingo. A lot of the ideas are just what you as a developer might do intuitively if you had to implement it. But the language tends to be a bit mathy and obscure. So, when you try to read something without understanding the lingo, it seems impenetrable. But once you know things like "quantizat…

I agree. I started Andrew NG's coursera course and it seemed pretty maths heavy and dry. I started reading through the tensorflow tutorial and couple other more hands on approaches. I get a better idea of what is actually happening in the second. Am I likely to need matrix multiplication if I start doing machine learning, or that the equivalent of writing a sort algorithm for a web dev - maybe useful to know the conc…

My experience so far is that it's important to know WHY you use linear algebra. The idea to me is that almost all input data passed to a machine learning algorithm should be transformed to a multi-dimensional array of doubles.

It's easier to write algorithms against this.

Since a lot of ML libraries use native libraries for linear algebra, you might see a lot of implementations that are written in terms of linear algebra operations. So, if you're trying to read the code and you don't understand what the operations do, it may be hard to grok.

So, yeah, I think some understanding of linear algebra is necessary. Because it's sort of the atomic set of operations underlying most ML you'll see. To read the code, you need to be able to read the linear algebra. But you probably don't need to go read a book on linear algebra. I tried that and it pulled me away from what I wanted to know. It might be enough to just understand the numpy docs.