I think your comment about disregarding the accelerating expansion of the universe is a key point. We can't actually move anywhere in space; reacheable space is constrained by our reference frame relative to the reference frame of some other part of the universe that we're accelerating away from. At a certain point, the parts of the universe become unreacheable, because we would have to accelerate faster than the speed of light in order to reach them.
If my understanding is correct, this same condition exists inside the photon limit of a black hole. Technically you're still in navigable space -- not inside the singularity yet -- and can move in any direction. But to actually escape the black hole would require accelerating faster than the speed of light.
Again, if my understanding is correct -- and I'm definitely not a phycisist by any stretch of the imagination -- our movement through time is exactly the same phenomena. We can slow our velocity through time (by moving through space instead), but we can't escape our local reference frame without moving faster than the speed of light. If we could exceed the speed of light, then we would be moving into spatial regions which are otherwise causally inaccessible to us; in other words, we'd be going backwards in time.
So: if we could go FTL, we could escape from black holes, visit parts of the universe beyond the locally-observable limit, and go backwards in time. I think (IANAP) that these are all describing precisely the same thing.
Dunno if this helps. The lawnmower analogy has definitely broken down by this point.
Also, it makes me think: if our experience of time is navigationally equivalent to the experience of space for someone getting sucked into a black hole, does that imply the existence of a higher-dimensional universe where ordinary, non-accelerating time is as fully navigable as our ordinary "non-accelerating" space? And in that higher-dimension universe, are we living near the surface of some kind of singularity? Do the inhabitants of that universe wonder about how sad it must be for poor creatures like us, forced to live on a time gradient which inexorably slopes in just one direction, the way we might commiserate the fate of those sucked into a black hole?