How to use Deep Learning when you have Limited Data
1–10 of 54 posts
Re: How to use Deep Learning when you have Limited Data
#2Machine Learning and AI seem to be in vogue but become tough to implement unless you have boatloads of data. We've personally had multiple frustrating experiences over the last ~7 years of trying to solve problems using ML. In almost all the cases we failed to ship due to lack of data. Transfer Learning is a major breakthrough in ML where companies with little data can also build state of the art models. Unfortunately not enough people know about it. We are trying to do our part to make it easier to use Transfer Learning as well as increase awareness about it.
Using Transfer Learning we can build a model to identify cats and dogs in images with a few (To make Transfer Learning easy we are building https://nanonets.ai that has multiple pretrained models that can be augmented with your data to create state of the art models. We are currently in the process of building our first few models. Image Labeling and Object Detection (in Images) work with a few Text based models coming up in the next few weeks.
Re: How to use Deep Learning when you have Limited Data
#3Another idea is one-shot learning using deep generative models. DeepMind had a paper on this last year:
https://arxiv.org/abs/1603.05106
Re: How to use Deep Learning when you have Limited Data
#4Machine Learning and AI seem to be in vogue but become tough to implement unless you have boatloads of data. We've personally had multiple frustrating experiences over the last ~7 years of trying to solve problems using ML. In almost all the cases we failed to ship due to lack of data. Transfer Learning is a major breakthrough in ML where companies with little data can also build state of the art models. Unfortunatel…
Is transfer learning really not widely known by people doing AI? In my field, computer vision, it is used by most of the papers in the past three years in CVPR, etc. All of the students that take either my deep learning or my computer vision courses have to do assignments on transfer learning with deep neural networks.
Re: How to use Deep Learning when you have Limited Data
#5Another idea is one-shot learning using deep generative models. DeepMind had a paper on this last year: https://arxiv.org/abs/1603.05106
Absolutely. One shot learning is the cutting edge research towards building more human like AI. However its still in early phases. We are trying to make transfer learning directly usable to people trying to solve problems which is proven today. Hopefully we will be able to do the same with one shot learning.
Re: How to use Deep Learning when you have Limited Data
#6Machine Learning and AI seem to be in vogue but become tough to implement unless you have boatloads of data. We've personally had multiple frustrating experiences over the last ~7 years of trying to solve problems using ML. In almost all the cases we failed to ship due to lack of data. Transfer Learning is a major breakthrough in ML where companies with little data can also build state of the art models. Unfortunatel…
Is transfer learning really not widely known by people doing AI? In my field, computer vision, it is used by most of the papers in the past three years in CVPR, etc. All of the students that take either my deep learning or my computer vision courses have to do assignments on transfer learning with deep neural networks.
Totally agree, everybody in the industry knows about it. However if you look at https://www.google.com/trends/explore?date=2014-01-01%202017... nobody outside seems to know. I might be wrong but a lot of people outside the ML community seem to be hesitant to using ml because they don't have enough data, trying to remove the misconception if it exists
Re: How to use Deep Learning when you have Limited Data
#7I think Deep Learning is very frustrating to work with at the moment. First, there is the problem of overfitting, which shows up typically after you've already been training for hours. So you have to tweak things (basically this is just guessing), and start from scratch. If your network has too many neurons, then overfitting may more easily occur, which is a weakness in the theory because how can more neurons cause more problems? Then there is the problem that if your data is somewhat different from your training data in what humans would call an insignificant way, your network may easily start to fail. For example, when doing image classification, and your images contain e.g. a watermark in the lower-left corner, suddenly your recognition may start failing. I've been able to use DL for some projects successfully, but for other projects it has been an outright failure with many invested hours of training and tweaking.
Re: How to use Deep Learning when you have Limited Data
#8I think Deep Learning is very frustrating to work with at the moment. First, there is the problem of overfitting, which shows up typically after you've already been training for hours. So you have to tweak things (basically this is just guessing), and start from scratch. If your network has too many neurons, then overfitting may more easily occur, which is a weakness in the theory because how can more neurons cause m…
This does seem to be holding back a lot of people from trying out DL in production. There are some advantages to using transfer learning in the cases you mentioned (eg models not generalizing for watermarks or differences in training and testing). Although there are still quite a few cases where the best pretraining and large data don't work. Two major areas of advancements (current research) are automatic model architecture selection and automatic parameter tuning to help in making DL more accessible.
Re: How to use Deep Learning when you have Limited Data
#9I think Deep Learning is very frustrating to work with at the moment. First, there is the problem of overfitting, which shows up typically after you've already been training for hours. So you have to tweak things (basically this is just guessing), and start from scratch. If your network has too many neurons, then overfitting may more easily occur, which is a weakness in the theory because how can more neurons cause m…
> which is a weakness in the theory because how can more neurons cause more problems?
In exactly the same way that adding more terms to a polynomial fit causes more problems. The is one of the most fundamental results in the theory of statistical learning in general; don't blame Deep Learning for it.
Re: How to use Deep Learning when you have Limited Data
#10I think Deep Learning is very frustrating to work with at the moment. First, there is the problem of overfitting, which shows up typically after you've already been training for hours. So you have to tweak things (basically this is just guessing), and start from scratch. If your network has too many neurons, then overfitting may more easily occur, which is a weakness in the theory because how can more neurons cause m…
> which is a weakness in the theory because how can more neurons cause more problems? In exactly the same way that adding more terms to a polynomial fit causes more problems. The is one of the most fundamental results in the theory of statistical learning in general; don't blame Deep Learning for it.
Yes I know, it was a rhetorical question. Imho, if having more parameters causes problems, then the system should simply not use those extra parameters. But the theory is not there yet.