Deep learning has a size problem
heartbeat.fritz.ai
Deep learning has a size problem
1–10 of 45 posts
Re: Deep learning has a size problem
#2It's not such a problem, except if you want to train from scratch a large model (NLP or CV), not if you want to fine-tune it for a related task. So one trained model can be reused many times. In general training data is scarce, only in a few situations it is abundant.
Re: Deep learning has a size problem
#3How did they use an elephant as cover image without mentioning von Neumann's famous and relevant quote: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk."
A great article on it: https://www.johndcook.com/blog/2011/06/21/how-to-fit-an-elep...
Re: Deep learning has a size problem
#4It doesn't parallelize well. Would be cool if you could loan CPU cycles on your phone or home computers while at work.
Re: Deep learning has a size problem
#5Deep learning doesn't parallelize well. Would be cool if you could loan CPU cycles on your phone or home computers while at work.
Re: Deep learning has a size problem
#6How did they use an elephant as cover image without mentioning von Neumann's famous and relevant quote: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk." A great article on it: https://www.johndcook.com/blog/2011/06/21/how-to-fit-an-elep...
That came up in The Dream Machine! Reading it now.
Re: Deep learning has a size problem
#7Seems to focus on reducing the size of existing models through optimization. Better would be to find ways to train smaller models to start with. Still interesting.
Re: Deep learning has a size problem
#8Seems to focus on reducing the size of existing models through optimization. Better would be to find ways to train smaller models to start with. Still interesting.
why would that be better?
Re: Deep learning has a size problem
#9Seems to focus on reducing the size of existing models through optimization. Better would be to find ways to train smaller models to start with. Still interesting.
why would that be better?
Compressing it means it may take less storage, but not having to look at it in the first place it the win. It simply takes time to process all the data. Less data: faster computation.
Re: Deep learning has a size problem
#10Deep learning doesn't parallelize well. Would be cool if you could loan CPU cycles on your phone or home computers while at work.
In what way does it not parallelize well? There are mounds of research in federated learning.