Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
theorangeduck.com
Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
1–10 of 23 posts
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#2Key quote from the conclusion: "Neural Networks are not just good for things we don't know how to solve, they can provide massive performance gains on problems we already know how to solve."
That quote is in reference to tasks such as physics simulation. There is an incredible GIF in the OP which shows a digital mannequin being manipulated, with its dress flowing in a hyper-realistic manner due to ML physics simulation. It would be uncanny to see that type of simulation combined with AR.
I'm curious to what extent ML physics simulation may be beneficial for self-driving cars. Generally, we as drivers know the physical properties of objects that we can collide with. Cars don't have that understanding, so they might "think" that colliding with a large paper bag is unacceptable. Stopping suddenly because of that paper bag may be fatal.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#3When I first learned about Kolmogorov complexity I understood it as the amount of symbols you have to use, from some specific vocabulary, to represent something.
It walks side-by-side with compression (and pigeon problems). Using Kolmogorov to improve ML in those physical examples means that the solution will be better to the specific case, not that there'll come a one-in-all solution to any kind of clothes animation.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#4TIL about Kolmogorov Randomness, and I'm a happier person.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#5There's an odd jump in this narrative. PCA is indeed a great technique, but the essay goes from PCA to neural nets without explaining why. PCA was around for a long time before NNs and there are fast incremental ways to do it. Why bother with a million-weight NN if PCA will do the job?
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#6Key quote from the conclusion: "Neural Networks are not just good for things we don't know how to solve, they can provide massive performance gains on problems we already know how to solve." That quote is in reference to tasks such as physics simulation. There is an incredible GIF in the OP which shows a digital mannequin being manipulated, with its dress flowing in a hyper-realistic manner due to ML physics simulati…
I don’t think the cloth simulation is based on ML, this looks like a normal physics-based solver. If I understand the article correctly the author just uses this as an example of complex behavior that could be learned using ML, but he/she doesn’t indicate that this was actually done using ML.
I know some papers that try to improve physics simulations with deep learning and I think it’s definitely possible, not sure though if it can really improve most physics-based simulations.
Physical models are already highly condensed and have the advantage of being interpretable, deep learning has a long way to go before it could be used as a replacement, IMO.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#7When I first learned about Kolmogorov complexity I understood it as the amount of symbols you have to use, from some specific vocabulary, to represent something. It walks side-by-side with compression (and pigeon problems). Using Kolmogorov to improve ML in those physical examples means that the solution will be better to the specific case, not that there'll come a one-in-all solution to any kind of clothes animation…
What's a pigeon problem? Googling for this term only tells me what to do if I have too many pigeons flying around my house.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#8Using Kolmogorov Complexity and then using PCA is not valid since you are approximating a solution and Kolmogorov Complexity is for exact solutions. Anyway perhaps there is, or should be defined, a signal/noise Kolmogorov Complexity measure, that is the shortest length of a program that computes an approximate solution within an epsilon distance of the true solution. Also since PCA is discussed, why not use SVD?
Edited: See (1)for some related ideas: A Safe Approximation for Kolmogorov Complexity
(1) https://link.springer.com/chapter/10.1007/978-3-319-11662-4_...
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#9There's an odd jump in this narrative. PCA is indeed a great technique, but the essay goes from PCA to neural nets without explaining why. PCA was around for a long time before NNs and there are fast incremental ways to do it. Why bother with a million-weight NN if PCA will do the job?
My understanding is that the neural network is used to predict the "magnitude" of each eigenvector/axe (extracted by using PCA) in order to reconstruct an approximation of the original behavior.
Re: Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
#10Using Kolmogorov Complexity and then using PCA is not valid since you are approximating a solution and Kolmogorov Complexity is for exact solutions. Anyway perhaps there is, or should be defined, a signal/noise Kolmogorov Complexity measure, that is the shortest length of a program that computes an approximate solution within an epsilon distance of the true solution. Also since PCA is discussed, why not use SVD? Edit…
Well stated. The author also misses two other critical points: (1) accuracy is a poor measure of quality for non-numeric, classification type problems, (2) increasing model complexity has an asymptote in order to prevent overfitting. You can’t arbitrarily increase the number of weights and expect that the NN will continue to improve.