The mathematical field of Differential Geometry can answer this question precisely:
https://en.wikipedia.org/wiki/Geodesic#Affine_geodesicsAn oblate spheroid is an example of a Riemannian manifold: a smooth object that looks like a plane (or, in general, any ℝ^n) locally, and has a way to measure angles between vectors in that local plane.
All Riemannian manifolds have an object called the Levi-Cevita connection, which defines how vectors in the local plane (tangent space) most naturally map to vectors in other tangent spaces in the immediate neighborhood.
Standing at a point on the Earth and looking in a certain direction gives us 1) a point on the manifold, and 2) a direction in that point's tangent space.
We then take an infinitesimally small step forward, and apply the Levi-Cevita connection to get from the old tangent space to the (infinitesimally nearby) new tangent space, and repeat. This defines an ordinary differential equation. Integrating the differential equation gives us a curve through the manifold.
Within some neighborhood of the initial point, this curve is a geodesic, i.e. the shortest path between the initial point and all subsequent points on the curve. This matches our typical intuition of "straight".
(Disclaimer: I am currently learning about this topic, but am not an expert.)
edit: https://en.wikipedia.org/wiki/Geodesics_on_an_ellipsoid goes into some interesting specifics about the results of this process on ellipsoids.