Something is wrong with navigation on my machine. Mouse movements are opposite my expectation (I guess just a preference), and the movements are so sensitive it jumps around a lot and I can't really navigate where I intend to. I'm on an iMac w/magic mouse.
The magic mouse is really bad with maps. I get the same problem with Google Map. As for the direction of the movement, think of it as pushing a globe and not moving a camera.
It's difficult to see how the magic mouse is responsible for the problems here. If I click and drag a point on a map, I expect that point to remain under the cursor. But – as described above – the map moves too fast.