When to assume neural networks can solve a problem
blog.cerebralab.com
When to assume neural networks can solve a problem
1–10 of 44 posts
Re: When to assume neural networks can solve a problem
#2I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might take to train a network, what kind of quality level is acceptable for inputs, etc.
Re: When to assume neural networks can solve a problem
#3This reminds me of an oldie, but goodie: http://thedailywtf.com/articles/No%2C_We_Need_a_Neural_Netwo...
Re: When to assume neural networks can solve a problem
#4I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might take to train a network, what kind of quality level is acceptable for inputs, etc.
PAC does not provide these types of estimates in most practical situations.
Re: When to assume neural networks can solve a problem
#5I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might take to train a network, what kind of quality level is acceptable for inputs, etc.
Correct me if I'm wrong, but doesn't PAC on real data usually just tells you that "it doesn't know the answer"? Like, it says the equivalent to: "I think this is the answer, but it might be miles off and I give it 50% it is even more than miles off." I found that it is able to give hard guarantees, by being very loose on the bounds. Do you have a real-world example where it actually gave something useful? I'm genuine curious, because I discarded it early on, but am willing to revisit it.
Re: When to assume neural networks can solve a problem
#6What is the current state of NN in the context of AI state of the art?
Re: When to assume neural networks can solve a problem
#7I don't understand why the author thinks that the PAC learning framework is not useful. PAC gives quick back-of-napkin math for estimating how long it might take to train a network, what kind of quality level is acceptable for inputs, etc.
I disagree, PAC bounds are usually way too loose to be useful in practice when it comes to NNs. I think part of the reason is that the way we measure model complexity for NNs does not correspond very well with their ability to generalize, as we've seen with recent papers like deep double descent [1].
Anecdotally, I was of the main authors of the AdaNet framework [2] where we used (approximate) Rademacher complexity (closely related to PAC) to bound the complexity of our learned ensemble models. We never got good results using the complexity measure and would basically turn it off for any practical problem we solved.
Re: When to assume neural networks can solve a problem
#8I've been hearing about deep learning revolutionizing "everything" for the past 8 years. So, can someone name me any significant impact it made on figuring out what to do about the recent epidemic? If not, I think it's worth reflecting on what value we get out of the technology that sucked up so much of our intellectual, financial and computational resources.
Re: When to assume neural networks can solve a problem
#9I've been hearing about deep learning revolutionizing "everything" for the past 8 years. So, can someone name me any significant impact it made on figuring out what to do about the recent epidemic? If not, I think it's worth reflecting on what value we get out of the technology that sucked up so much of our intellectual, financial and computational resources.
I've been hearing about modern JavaScript development revolutionizing "everything" for the past 8 years. So, can someone name me any significant impact it made on figuring out what to do about the recent epidemic? If not, I think it's worth reflecting on what value we get out of the technology that sucked up so much of our intellectual, financial and computational resources.
Re: When to assume neural networks can solve a problem
#10This reminds me of an oldie, but goodie: http://thedailywtf.com/articles/No%2C_We_Need_a_Neural_Netwo...
Wow, I had never read this gem before. Made me laugh, thanks for sharing this piece! I'm surprised they did not ask him to log everything in a blockchain :)