Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I Became a Machine Learning Practitioner

blog.gregbrockman.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: How I Became a Machine Learning Practitioner

#21

Post by dhairya »

jph00 wrote: post #14
dhairya wrote: post #10

Earlier quoted context omitted.

So there is a difference between ML research and application. Being a practitioner doesn't require deep math knowledge that perhaps research would. Jeremy Howard's fastai course is a great example of how someone with a solid programming background can effectively transition into being a deep learning practitioner. Given that production ml and deep learning is still the wild west, as a practioner, you can contribute a…

Also, note that Greg's goal was to contribute to OpenAI's flagship project. That's a rather ambitious goal! Also, most folks I know that are making practical deep learning contributions are doing so by combining their pre-existing domain expertise with their new deep learning skills. E.g. a journalist analyzing a large corpus of text for a story, or an oil&gas analyst building models from well plots, etc.

as a side note, I love that you highlight regex in your new NLP course. There is an inherent tension between the probabilistic nature of models and the need for deterministic outputs in most production settings. Often if we can uncover linguistics rules or regex patterns that guarantee minimal precision (or as our VP puts it - don't look stupid), we'll eschew the model in the short term or use the model to augment the rules.

Also I really appreciated that on of the training goals for ULMfit was to be trainable on a single gpu. With these large-capacity models, training is getting crazy expensive and out of hand. Any chance that your future work will still keep the single gpu training goal?

Re: How I Became a Machine Learning Practitioner

#23

Post by rhacker »

klipt wrote: post #18
jorblumesea wrote: post #13

Does anyone not want to become a ML engineer? Is this the future, and will we even have a choice or else be out of a job?

There's plenty of non ML software engineering to be done. Anecdotally a large proportion of interns want an "ML project", but only a small percentage of teams looking for interns are offering one. Too many people going into ML could skew the supply/demand into making it a worse job option (more work, less pay), like game programming or academia.

The question is this: 10 years from now when the top job requirements list ML - are you going to be ready or out of the game?

Re: How I Became a Machine Learning Practitioner

#24

Post by sevensor »

dhairya wrote: post #10
copperx wrote: post #3

Earlier quoted context omitted.

Studying Math at Harvard/MIT certainly puts you in a different category than the average software engineer. And if ML was still challenging to Greg, it is honestly a bit discouraging.

So there is a difference between ML research and application. Being a practitioner doesn't require deep math knowledge that perhaps research would. Jeremy Howard's fastai course is a great example of how someone with a solid programming background can effectively transition into being a deep learning practitioner. Given that production ml and deep learning is still the wild west, as a practioner, you can contribute a…

I recently got the assignment to "do ML" on some data. I hadn't done anything in the area before, and a couple of things surprised me:

1. Most of your time is spent transforming data. Very little is spent building models.

2. Most of the eye-grabbing stuff that makes headlines is inapplicable. My application involves decisions that are expensive and can be safety critical. The models themselves have to be simple enough to be reasoned about, or they're no use.

You might argue that this means what I'm actually doing is statistics.

Re: How I Became a Machine Learning Practitioner

#25

Post by ilyasut »

mychael wrote: post #22

Congrats on your cool life, your ivy league education, your CTO role at OpenAI and all the access that provides. You've done it! Also thanks for telling us how you became a practitioner. It's definitely relatable and not a humble brag at all.

Comments like this are what make HN great and are not at all toxic!

Re: How I Became a Machine Learning Practitioner

#26

Post by mnky9800n »

sevensor wrote: post #24
dhairya wrote: post #10

Earlier quoted context omitted.

So there is a difference between ML research and application. Being a practitioner doesn't require deep math knowledge that perhaps research would. Jeremy Howard's fastai course is a great example of how someone with a solid programming background can effectively transition into being a deep learning practitioner. Given that production ml and deep learning is still the wild west, as a practioner, you can contribute a…

I recently got the assignment to "do ML" on some data. I hadn't done anything in the area before, and a couple of things surprised me: 1. Most of your time is spent transforming data. Very little is spent building models. 2. Most of the eye-grabbing stuff that makes headlines is inapplicable. My application involves decisions that are expensive and can be safety critical. The models themselves have to be simple enoug…

> You might argue that this means what I'm actually doing is statistics.

whats the difference?

Re: How I Became a Machine Learning Practitioner

#27

Post by dhairya »

streetcat1 wrote: post #12
dhairya wrote: post #11

Earlier quoted context omitted.

No worries, fair question. It worth noting that my job is not data analysis. So I do use data analysis to evaluate our metrics and model performance. Really none of it is really automatable. I'm working developing NLP features for our product (question answering, search, neural machine translation, dialog, etc). Our customer data is diverse, in different formats, and thier use cases are all distinct. So most of my wo…

Thanks. I do assume that the data format is different (alas I also assume that they are all some sort of a text file with known fields and types). But after you setup the dataset definition and defined the schema, the rest can be based on neural search? Moreover, isn't there a state of the art architecture for each of the task. E.g. Seq2Seq for machine translation. Can you just use that as a base line, and let the NA…

Happy to talk more offline, my email is my profile. The short answer is no because there are more complexities involved - both related to our specific use cases but really natural language in general. If that were the case NLP would be solved and any company that could exist would already. From my experience, I'm not sure where the line is between choosing the right model vs having the right data solves most problems. There have been novel architecture developments like rnns and lstms that have shown well to support certain domains. New architecture are developing each year and the space moves very quickly. On the flip side, having pedabtyes of data (like BERT or OpenGPT) and simpler architectures is also powerful but prohibitive to everyone that is not Google or state government. The real answer is probably somewhere in between and whiles it's unsolved, there is work for me to do. That being said, our strategic philosophy is to make our AI a commodity so that we can differentiate ourselves on other features.

Most of our problem don't cleanly map to existing NLP tasks. State of the art often isn't as high as you think in many tasks. For example, the machine translation in relation to beta feature we're building that lets you ask the question of arbitrary single tables (kind of like wiki-tables) but we don't the know the schemas in advance or the questions the user may ask about. Outside of having the issue of having quality annotated data (which we often don't - cold start problem), we need to do more than simple model tuning. It requires building custom architectures.

But even when you consider known tasks, state of the art models do not often produce those same results on real-world data. If you put aside data quality issues (which is another huge challenge for us), in the context of question answering, the training data rarely captures the distribution of the natural language in the wild. People ask questions differently and use language that doesn't match the content in our knowledge base.

I could go on. But short answer, it's not as straightforward as you think. Even at google scale, machine learning is not solved. For everyone else with fewer data and domain-specific use cases, it's even harder.

Re: How I Became a Machine Learning Practitioner

#28

Post by FridgeSeal »

streetcat1 wrote: post #12
dhairya wrote: post #11

Earlier quoted context omitted.

No worries, fair question. It worth noting that my job is not data analysis. So I do use data analysis to evaluate our metrics and model performance. Really none of it is really automatable. I'm working developing NLP features for our product (question answering, search, neural machine translation, dialog, etc). Our customer data is diverse, in different formats, and thier use cases are all distinct. So most of my wo…

Thanks. I do assume that the data format is different (alas I also assume that they are all some sort of a text file with known fields and types). But after you setup the dataset definition and defined the schema, the rest can be based on neural search? Moreover, isn't there a state of the art architecture for each of the task. E.g. Seq2Seq for machine translation. Can you just use that as a base line, and let the NA…

Sure, provided you have enough data to feed a neutral net, and the problem is well suited to it and you don’t mind giving up huge chunks of explainability.

I recently replaced a classifier at work that was using a neural net with a decision tree and some hand chosen features. It performs a bit better, it takes way less time to train and it’s significantly more explainable: my teammates asked why it sometimes miss-classifies a certain edge case, and because the features and model properties were so easy to understand, fixing the issue was a couple of hours work and not a case of “who knows”.

Re: How I Became a Machine Learning Practitioner

#29

Post by klipt »

mnky9800n wrote: post #26
sevensor wrote: post #24

Earlier quoted context omitted.

I recently got the assignment to "do ML" on some data. I hadn't done anything in the area before, and a couple of things surprised me: 1. Most of your time is spent transforming data. Very little is spent building models. 2. Most of the eye-grabbing stuff that makes headlines is inapplicable. My application involves decisions that are expensive and can be safety critical. The models themselves have to be simple enoug…

> You might argue that this means what I'm actually doing is statistics. whats the difference?

ML conferences have way bigger budgets.

Re: How I Became a Machine Learning Practitioner

#30

Post by TrackerFF »

I think it's doable if you're at the right place and have enough opportunities around you, and something to show for. I feel that companies and startups around tech hubs are more willing to give someone a chance, and look through the formality, if you manage to convince / impress them.

Where I live, far away from tech, it's almost impossible to land a job in ML / AI / DS unless you have a (minimum) Masters degree in something relevant. Preferably a Ph.D and solid experience to show for - I know because I work in the field, and lots of F500 dinosaurs are just now waking up. But are also unfortunately clinging to their old ways of hiring people.

Schools all over are also picking up slack, starting to offer specialized graduate degrees in those domains. When I got my degree in ML, it was a sub-field at my schools engineering department, mixed up with signal processing and control theory groups.

When first trying to get a job, the main problem was to explain what I actually could bring and do, and a lot of the recruiters or managers had no idea what Machine Learning was. Then you said "It's basically Artificial Intelligence" and, and they were instantly wooed.

Post reply on HN