So for instance, a typical NLP algorithm (although not GPT-3, IIRC) might represent a word as a 500-float-long vector, which is the same as saying the algorithm considers each word as a point in 500-dimensional space. This turns out to have weirdly useful properties, to the point where directions in this 500-dimensional space start to have semantic correspondences (e.g. [0], still one of the coolest things in ML, IMHO.) You can't do the same trick with a 3D space- the algorithm doesn't have enough to work with when all it knows about a word is three numbers.
Another cool example- in gradient descent, you're constantly trying to find the lowest point in a "fitness landscape"; in a 3D landscape, you might easily find yourself in a "valley" where every direction is worse than you currently are (a local minima), and you won't know where to go. In a 500D landscape, it's unlikely that you'll find yourself in a valley where all 500 available directions lead somewhere worse. So the algorithm will be much less likely to get stuck, and this effect gets more robust the more dimensions you have.
[0] https://colah.github.io/posts/2014-07-NLP-RNNs-Representatio...