Live data from Hacker News

Facebook AI Research Team Open Source DeepMask and SharpMask

code.facebook.com

41–50 of 89 posts

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#41
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

fwiw, over 4 years ago, I quit a risk quant job at an investment bank and joined Twitter. When I saw all the sophisticated ML systems they had deployed, I asked the same question - How do I, a person of average intelligence with zero exposure to ML, contribute ? Is there anything I can do to get involved?

The answer I got was rather unpleasant and won't please everybody. No you cannot contribute. At least not in a direct meaningful fashion. You can be a waterboy. So that's what I did for 2 years. I wrote a lot of ETL jobs using custom Scala DSLs, that fed the input to these ML jobs. It was a total waste of time. Sure I learnt map reduce and Hadoop and all that jazz, but end of the day, I wasn't doing ML. I was doing ancillary tasks. These tasks no doubt have some economic worth, because I was getting paid. But no company is going to let you do the ML when they have 100s of ML PhDs on their payroll and you aren't one of them. So you just do the data prep, or do ETL, or do data viz, or crunch some numbers aka BI, and convince yourself you are doing real ML. This went on for a while. Finally I couldn't put up with this farce and quit. What worked for me personally was finding a very small company with a tiny data science department, that was headed by an ML PhD who was ready to mentor me, tell me which papers to read, get me to start working on my own papers, get me to build ML systems for image recognition on company time, get me to read textbooks and present topics...it was all quite painful and very humbling, but I learnt a shit ton of stuff. So my frank suggestion is to be brutally honest with yourself. You aren't going to get from here to there by hanging around on coursera or writing ETL. This stuff is seriously hard. If you want to make a genuine contribution, be willing to put in serious time - and that means literally stopping whatever shit you are doing now as a webdev/back-end dev/ETL dev/data-eng etc. Those ancillary tasks won't get you anywhere. Buckle down and do the real deal. You'll thank yourself one day if you did.

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#42
post #41
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

fwiw, over 4 years ago, I quit a risk quant job at an investment bank and joined Twitter. When I saw all the sophisticated ML systems they had deployed, I asked the same question - How do I, a person of average intelligence with zero exposure to ML, contribute ? Is there anything I can do to get involved? The answer I got was rather unpleasant and won't please everybody. No you cannot contribute. At least not in a di…

While I really appreciate the realism in your post, I wouldn't call webdev/back-end dev shit though. After all, that's what brought us here in the first place and are still heavily needed.

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#43
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

I have a pile of links for getting started with DL in my comment history you can use: https://news.ycombinator.com/item?id=10676455

What really helped advance my understanding from zero to knowledgeable novice was rewriting some existing code line by line (using expanded variable names and comments), and thinking about each line and what it does as you go. It's the software development equivalent of Hunter S. Thompson re-typing The Great Gatsby just to get the feel of writing a great novel. Here's one I did based on Denny Britz's tutorial:

Britz's Original: http://www.wildml.com/2015/09/implementing-a-neural-network-...

My version: https://gist.github.com/sthware/c47824c116e6a61a56d9

HTH

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#44
post #41
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

fwiw, over 4 years ago, I quit a risk quant job at an investment bank and joined Twitter. When I saw all the sophisticated ML systems they had deployed, I asked the same question - How do I, a person of average intelligence with zero exposure to ML, contribute ? Is there anything I can do to get involved? The answer I got was rather unpleasant and won't please everybody. No you cannot contribute. At least not in a di…

Doing the dev is the only way, I would recommend though reading the Hinton dropout paper and trying to implement a non full featured Neural Network framework to implement those ideas. Then pick papers to implement in your toy framework. Once you have done this a few times, you will get a really good idea how all this stuff works and ideas to extend things. It also REALLY helps if you have a very strong linear algebra background, so if your weak there do lots of reading and practice getting those skills up to snuff. I may not be the best example, because I worked in computer graphics heavily over my career. My suggestion though is to try and apply DNN's and ML techniques in somewhere you have domain experience. It really will help a lot with the intuition part since you have valuable domain experience to give you ideas to try.

This is a path I took around 4 years ago, and I have built some seriously valuable stuff in that time.

Also I would personally say to not discount the value of backend stuff, doing scalable ML processing is not off the shelf and lots of value can be created by making scalable reusable systems that can run machine learning models. Most of those ML PHd's can't (or don't want to) build scalable distributed systems. If you can train experiments faster, or actually do the work to make an end to end system work outside the lab, you can make real measurable contributions.

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#45

Earlier quoted context omitted.

> Is there any way for those of us with average intelligence to contribute to tools like this? The people doing this aren't magical geniuses; they've just put the time and work into the subject and have been able to get themselves into a position they can do this all day surrounded by others they can collaborate with. As with most human endeavors, the trick is to just get started, and not get frustrated and give up w…

> The people doing this aren't magical geniuses Says the person named 'magicalist'. :)

double upvote

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#46
post #24

How 'deep' are the networks used in something like DeepMask, and how does it compare with the number of layers of the human brain?

For example, ResNet from 2015 had 152 layers.

A real neuron takes in the order of 10ms to integrate and fire to the next neuron. Many subconscious reactions take less than 1 sec, which leaves time to a chain of length less than 100. Note that those neurons are not strictly arranged in layers.

The human visual cortex has 10^12 synapses [1]. One popular 2015 deep learning net (ResNet 152-layers) used 10^12 FLOPs to classify objects in one image (but less weights.)

In terms of depth, we're there. In terms of breadth, it will take several years. But the brain does things very differently. For example, it has top-down signals during "prediction."

[1] http://www.ncbi.nlm.nih.gov/pubmed/7244322

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#47
post #24

How 'deep' are the networks used in something like DeepMask, and how does it compare with the number of layers of the human brain?

For example, ResNet from 2015 had 152 layers. A real neuron takes in the order of 10ms to integrate and fire to the next neuron. Many subconscious reactions take less than 1 sec, which leaves time to a chain of length less than 100. Note that those neurons are not strictly arranged in layers. The human visual cortex has 10^12 synapses [1]. One popular 2015 deep learning net (ResNet 152-layers) used 10^12 FLOPs to cla…

"It will take several years."

A Noob question:

If it reaches the "breadth" of human brain, how close will that to the "Skynet becomes self-aware" moment.

What would it tell us about human, our society after it study, analyze millions, billions hours of FB, youtube videos?

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#48
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

`Hacker's guide to Neural Networks` http://karpathy.github.io/neuralnets/

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#49
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

dxbydt nailed it on the head: https://news.ycombinator.com/item?id=12362642

If you want to make actual contributions to the machine learning part of machine learning (i.e. not building infrastructure and pipelines), then you need to get your math into a really good state. That comes first before you can even hope to begin to learn ML. That means acing probability and linear algebra, which is most of what ML is based off of.

If you have no experience with probability or linear algebra, it'll take you at least a year of solid studying to get up to speed.

Then, you can start taking ML courses. That will take you another year to really understand well.

At that point, as dxbydt said, you should find a ML PhD who is willing to mentor you. Otherwise, you will not be able to make meaningful contributions, since there are thousands of ML PhDs out there who are much better skilled than you are, and companies would rather hire them than you.

So the ultimate answer is: yes, with a ton of sacrifices and years of work. The only question for you to answer is whether or not you want to make those sacrifices and put in that work.

Re: Facebook AI Research Team Open Source DeepMask and SharpMask

#50
post #6

Is there any way for those of us with average intelligence to contribute to tools like this? I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

You would need to join a community of people who can show you how to contribute to these projects. There are a lot of very active open-source communities, including OpenAI and Deeplearning4j, to name but two.[0][1]

People in those communities can't point you in the right direction, if you show you're willing to read the docs and do the work of learning. One thing you will find is that AI requires a lot of non-AI components to work. That is, you may be able to help an open-source project develop its UI, or improve its datapipeline, or any number of other useful things. By working on the edges of such a project, you'll learn the lingo and grasp some of the basic ideas. From there, you can start working on the math, etc.

Bottom line, there are a lot of ways to get involved with AI.

[0] https://gitter.im/openai/gym

[1] https://gitter.im/deeplearning4j/deeplearning4j

[2] http://deeplearning4j.org/deeplearningforbeginners.html

Post reply on HN