Dive into Deep Learning • Hacker News Board

Re: Dive into Deep Learning

#11

Post by throwlaplace » Fri, Jan 03, 2020, 8:09 PM UTC

>What's a labelling tool that non-technical users can use intuitively?

i haven't used it but microsoft has this

https://github.com/microsoft/VoTT

>"If we spend all this time and money labelling footage, how well is the going to work?"

"not well at all because we don't have facebook/google scale training data. let's try to figure out a conventional way to do it". for the supervisors problem i would recommend bluetooth beacons.

Re: Dive into Deep Learning

#12

Post by ssivark » Fri, Jan 03, 2020, 8:22 PM UTC

Awesome summary. Welcome to some lessons/truths (circa 2019 state of technology):

1. Deep learning (by itself) is often a shitty solution. It takes a lot of fiddling with not just the models, but also the training data — to get anything useful. Often the data generation team/effort becomes larger than the model-building effort.

2. It is hopeless to use neural networks as an end-to-end solution. This example will involve studying whether detections are correlated/independent in neighboring frames... whether information can be pooled across frames... whether you can use that to build a robust real-time of the scene of interest, etc. That will involve lots of judicious software system design using broader ideas from ML / statistical reasoning.

This is why I find it hopelessly misleading to tell people to just find tutorials with TensorFlow/Pytorch and get started. You really need to understand what’s going on to be able to build useful systems.

That’s apart from all the thorny ethical questions raised by monitoring humans.

Re: Dive into Deep Learning

#13

Post by meritt » Fri, Jan 03, 2020, 8:22 PM UTC

Why is this a machine learning problem? Does your factory not have keycard access? Or just require your supervisors to carry some sort of RFID/BLE tracking device. These are well-solved problems.

Re: Dive into Deep Learning

#14

Post by fareesh » Fri, Jan 03, 2020, 8:26 PM UTC

I wrote random numbers for the sake of narrating a scenario but yeah I suppose you could do Supervisor Present Y/N for 180 frame chunks @30fps and pick up the value per minute

Re: Dive into Deep Learning

#15

Post by fxtentacle » Fri, Jan 03, 2020, 8:26 PM UTC

Most AI stuff is just horribly over-hyped, so the sad truth might be that what you are seeing is the state of the art and nobody else has found a better way yet.

As a practical example, figuring out where a given pixel moves from one video frame to the next one, when working on real-world videos, the best known algorithms get about 50% of the pixels correct. With clever filtering, you can maybe bump that to 60 or 70%, but in any case you will be left with a 30%+ error rate.

NVIDIA / Google / Microsoft / Amazon will tell you that you need to buy or rent more GPUs or Cloud GPU servers and do more training with more data. And there's plenty of companies in cheap labor countries offering to do your data annotation at a very reasonable rate. But both of them are just trying to sell to you. They don't care if it will solve your problem, as long as you're feeling hopeful enough to buy their stuff.

Judging from the bad results that even Google / Facebook / NVIDIA show at benchmarks, having a near-unlimited budget is still not enough to make ML work nicely.

Oh and for these image classification networks like YOLO, they have their own flavor of problems: https://www.inverse.com/article/56914-a-google-algorithm-was...

Re: Dive into Deep Learning

#16

Post by sickcodebruh » Fri, Jan 03, 2020, 8:26 PM UTC

Have you tried fast.ai's Practical Deep Learning For Coders? https://course.fast.ai/ I think it's great for answering many of the exact questions you have.

I was able to answer my own versions of many of those questions after the first few video lessons. It demonstrated to us that our data is a great fit for machine learning. I didn't feel comfortable turning my experiments into something production-worthy but I feel confident enough to at least have conversations about it and sketch out a possible plan for what a contractor could work on this year.

Re: Dive into Deep Learning

#17

Post by salty_biscuits » Fri, Jan 03, 2020, 8:26 PM UTC

You need to start from what sort of accuracy do you need for the task from a business perspective (including what is acceptable in terms of false positives and false negatives). Just back of the envelope stuff. You have a rough idea of the "I copied stuff other people has done rate" and the "I spent few a days mucking about rate". This stuff always follows a logistic curve with time, starting at your first rate and asymptotically going to high 90%. Use this to get a ball park estimate of how long it will take / cost. If the accuracy required is close to 100% you can probably give up straight away. For things like this that I have done in the past, a good mental model has been if it isn't worth "manually automating" the task (i.e. paying someone somewhere to watch a webcam and send the email so you always have the end product and you eventually get labeled data as a byproduct) it might not be worth trying to automate it.

Re: Dive into Deep Learning

#18

Post by fareesh » Fri, Jan 03, 2020, 8:31 PM UTC

There seem to be a lot of courses in this space - I'll give this one a try since you're recommending it. Most of them seem to focus more on the theoretical / math aspects of stuff, which is quite interesting but I find it more interesting to implement these things and solve real-world problems.

Re: Dive into Deep Learning

#19

Post by proc0 » Fri, Jan 03, 2020, 8:32 PM UTC

Isn't this a human learning problem? Just tell your supervisors to be aware of their counterpart on the floor, at all times?

Re: Dive into Deep Learning

#20

Post by zmmmmm » Fri, Jan 03, 2020, 8:32 PM UTC

The shocking thing that at least I ran into is the sheer quantity of training data you really need. The large companies doing this successfully are using utterly gigantic libraries of training data that are beyond anything others could ever come up with. It really brought home to me what a blunt intstrument deep learning really is.