For anyone interested, i have a whole bunch of links on Deep-Learning and much more: http://tagly.azurewebsites.net/Home/ByTag?Name=deep-learning
How to use Deep Learning when you have Limited Data
41–50 of 54 posts
Re: How to use Deep Learning when you have Limited Data
#42Earlier quoted context omitted.
Is transfer learning really not widely known by people doing AI? In my field, computer vision, it is used by most of the papers in the past three years in CVPR, etc. All of the students that take either my deep learning or my computer vision courses have to do assignments on transfer learning with deep neural networks.
It is probably transfer learning has become more of a technique used for training neural network models (such as Adam being one of the most commonly used optimizer) whereas most of the excitement stems from new and/or complex neural network architectures rather than the technique or tools that made training those neural networks possible.
Transfer learning in ML refers to the general idea of taking a model trained for one domain and applying it to another. However this article seems to only focus on feature mapping, ie. breaking down images into features using hidden layers of ImageNet models. In this case, the pretrained model is only acting as a feature extractor because it is not trained to maximize the embedded distance between the classes you are trying to differentiate.
Re: How to use Deep Learning when you have Limited Data
#43Thanks for this write up. I'm not in the ML field, but do follow it a bit and didn't know anything about it.
I also really like your business model. I had argued with potential entrepreneurs and friends that ai is becoming a commodity. However, your business model has a potential for building a network effect of data on top of ai. Presumably becoming more valuable with time. I do think though, you probably best solve this for a specific vertical first as your go to market strategy.
Re: How to use Deep Learning when you have Limited Data
#44Earlier quoted context omitted.
This is Deep Learning 101 level material, rather than an advanced insider technique. It's even listed in the Tensorflow tutorials: https://codelabs.developers.google.com/codelabs/tensorflow-f...
While I agree that you could build a demo in Deep Learning 101 that could work for some small set of examples, I disagree that this is 101 level material. Facebook just released: https://techcrunch.com/2017/02/02/facebooks-ai-unlocks-the-a... You could also call this Deep Learning 101. But it really isn't because building a usable platform that works at scale actually delivers performance and solves problems is a lot…
Just poked around a bit with your API, and the learning with just 25 samples is impressive! And the getting training samples from the web is a great touch. But that 25 sample number seems too low for classes that are "closer" together? How do you quantify if you have done a good job on training or if you need some varied samples?
Re: How to use Deep Learning when you have Limited Data
#45Earlier quoted context omitted.
It is probably transfer learning has become more of a technique used for training neural network models (such as Adam being one of the most commonly used optimizer) whereas most of the excitement stems from new and/or complex neural network architectures rather than the technique or tools that made training those neural networks possible.
Transfer learning in ML refers to the general idea of taking a model trained for one domain and applying it to another. However this article seems to only focus on feature mapping, ie. breaking down images into features using hidden layers of ImageNet models. In this case, the pretrained model is only acting as a feature extractor because it is not trained to maximize the embedded distance between the classes you are…
Your right the article doesn't do justice to all of the transfer learning techniques. I cut out large portions of the original draft for brevity. Will write a longer post referencing all possible techniques, hopefully it won't turn into a review paper.
Re: How to use Deep Learning when you have Limited Data
#46Earlier quoted context omitted.
It is probably transfer learning has become more of a technique used for training neural network models (such as Adam being one of the most commonly used optimizer) whereas most of the excitement stems from new and/or complex neural network architectures rather than the technique or tools that made training those neural networks possible.
Transfer learning in ML refers to the general idea of taking a model trained for one domain and applying it to another. However this article seems to only focus on feature mapping, ie. breaking down images into features using hidden layers of ImageNet models. In this case, the pretrained model is only acting as a feature extractor because it is not trained to maximize the embedded distance between the classes you are…
[deleted]
Re: How to use Deep Learning when you have Limited Data
#47Earlier quoted context omitted.
While I agree that you could build a demo in Deep Learning 101 that could work for some small set of examples, I disagree that this is 101 level material. Facebook just released: https://techcrunch.com/2017/02/02/facebooks-ai-unlocks-the-a... You could also call this Deep Learning 101. But it really isn't because building a usable platform that works at scale actually delivers performance and solves problems is a lot…
Just poked around a bit with your API, and the learning with just 25 samples is impressive! And the getting training samples from the web is a great touch. But that 25 sample number seems too low for classes that are "closer" together? How do you quantify if you have done a good job on training or if you need some varied samples?
Thanks for trying us out! Internally we have validation metrics which have a number that says how good a model is at class separation. One naive way to do this entropy (-plogp). We are planning on exposing this to users soon. So once you create a model you'll receive feedback as to how "good" we think it is. In case it's not working well we might ask you for more data (we hope we don't need to do this too frequently)
Re: How to use Deep Learning when you have Limited Data
#48Machine Learning and AI seem to be in vogue but become tough to implement unless you have boatloads of data. We've personally had multiple frustrating experiences over the last ~7 years of trying to solve problems using ML. In almost all the cases we failed to ship due to lack of data. Transfer Learning is a major breakthrough in ML where companies with little data can also build state of the art models. Unfortunatel…
This is Deep Learning 101 level material, rather than an advanced insider technique. It's even listed in the Tensorflow tutorials: https://codelabs.developers.google.com/codelabs/tensorflow-f...
Another great example is the last post on the keras blog [1] "Using pre-trained word embeddings in a Keras model". You take advantage of a large pre-trained network for a text classification task.
The OpenFace face recognition library also offers this technique. You take advantage of their large pre-trained network for face embedding: transforming a face into features distinct enough for classification. You then train another few layers for recognizing your own samples.
1: https://blog.keras.io/using-pre-trained-word-embeddings-in-a...
Re: How to use Deep Learning when you have Limited Data
#49Another idea is one-shot learning using deep generative models. DeepMind had a paper on this last year: https://arxiv.org/abs/1603.05106
Not to be a pedant, but I think the DeepMind paper is actually an example of one-shot generalization, but not learning. From the paper:
> Another important consideration is that, while our models can perform one-shot generalization, they do not perform one-shot learning. One-shot learning requires that a model is updated after the presentation of each new input, e.g., like the non-parametric models used by Lake et al. (2015) or Salakhutdinov et al. (2013). Parametric models such as ours require a gradient update of the parameters, which we do not do. Instead, our model performs a type of one-shot inference that during test time can perform inferential tasks on new data points, such as missing data completion, new exemplar generation, or analogical sampling, but does not learn from these points. This distinction between one-shot learning and inference is important and affects how such models can be used.
Re: How to use Deep Learning when you have Limited Data
#50Machine Learning and AI seem to be in vogue but become tough to implement unless you have boatloads of data. We've personally had multiple frustrating experiences over the last ~7 years of trying to solve problems using ML. In almost all the cases we failed to ship due to lack of data. Transfer Learning is a major breakthrough in ML where companies with little data can also build state of the art models. Unfortunatel…
what do your false positives and false negatives rates look like with this approach compared to other classifiers?