Counterintuitive Properties of High Dimensional Space
marckhoury.github.io
Counterintuitive Properties of High Dimensional Space
1–10 of 68 posts
Re: Counterintuitive Properties of High Dimensional Space
#2Great article! I've been thinking about this on and off recently, as I wonder if we might be having intuitive issues when it comes to gradient descent optimization.
3 Blue,1 Brown had a video recently that kicked off my head scratching and is a great complement to your article: https://www.youtube.com/watch?v=zwAD6dRSVyI
Re: Counterintuitive Properties of High Dimensional Space
#3This was the first application of the Monte Carlo numerical method I've done in college. Take an N-dimensional cube with volume = 1. Inscribe an N-sphere in it. Calculate the volume of the N-sphere as a function of N.
I was too lazy to do the strict proof, so I sprayed it with lots of Monte Carlo bullets. It's like a page of code in any language. It turns out, as the article says, the volume of the N-sphere keeps getting smaller and smaller as N increases. In higher dimensions, there's a lot more volume in the corners of the N-cube. It did seem like the N-sphere was shrinking down to nothing in spaces with lots of dimensions.
Seems obvious after you read the article and look at the diagrams, but back then I had to think about it for a while. I thought my implementation was wrong somehow, but eventually I realized what was going on. Pretty amazing stuff.
Re: Counterintuitive Properties of High Dimensional Space
#4Just today on numberphile channel they showed how to pass circle through smaller square hole by bending it in higher (3rd) dimension: https://m.youtube.com/watch?v=AvFNCNOyZeE
Re: Counterintuitive Properties of High Dimensional Space
#5Just today on numberphile channel they showed how to pass circle through smaller square hole by bending it in higher (3rd) dimension: https://m.youtube.com/watch?v=AvFNCNOyZeE
That short little video gave me a lot of pleasure!
Re: Counterintuitive Properties of High Dimensional Space
#6This was the first application of the Monte Carlo numerical method I've done in college. Take an N-dimensional cube with volume = 1. Inscribe an N-sphere in it. Calculate the volume of the N-sphere as a function of N. I was too lazy to do the strict proof, so I sprayed it with lots of Monte Carlo bullets. It's like a page of code in any language. It turns out, as the article says, the volume of the N-sphere keeps get…
I was reminded of this:
https://mikesmathpage.wordpress.com/2016/04/29/a-strange-pro...
which I solved with this F# script, while learning F# https://gist.github.com/jackmott/bec1e4c7e84904702bac1dae97c...
Re: Counterintuitive Properties of High Dimensional Space
#7Great article! I've been thinking about this on and off recently, as I wonder if we might be having intuitive issues when it comes to gradient descent optimization. 3 Blue,1 Brown had a video recently that kicked off my head scratching and is a great complement to your article: https://www.youtube.com/watch?v=zwAD6dRSVyI
Thank you! It's funny you should mention that because I've been thinking about continuous optimization a lot lately.
That's a great video, I really like his slider method for understanding the coordinates. Thanks for linking it!
Re: Counterintuitive Properties of High Dimensional Space
#8Just today on numberphile channel they showed how to pass circle through smaller square hole by bending it in higher (3rd) dimension: https://m.youtube.com/watch?v=AvFNCNOyZeE
The did a numberphile video on this exact topic a few months ago:
https://www.youtube.com/watch?v=mceaM2_zQd8
Re: Counterintuitive Properties of High Dimensional Space
#9The sphere drawing in figure 7...
People seem to miss this subtelty, the spikeness is an artifact of the coordinate system. Rotate the basis vectors and the spikes move with them.
Re: Counterintuitive Properties of High Dimensional Space
#10from the article:
The volume of the unit d-sphere goes to 0 as d grows! A high dimensional unit sphere encloses almost no volume!
Maybe I'm nitpicking, but this interpretation is not accurate IMO. Volume of N-dimensional sphere is measured is different units than that of (N-1)-dimensional sphere. E.g. one is m^5, and another is m^4. Comparing values measured in different units is not the best idea. It would be better worded as : the ratio of volume of N-dimensional sphere to the volume of same-dimensional cube approaches zero when N goes to infinity. But this is not a counter-intuitive statement.