I earned a masters in physics about 15 years ago and started in industry as a software developer. Over time I started working on more analytic applications, basically because I could understand the math, and for the last six years I've worked full time as a data scientist. My physics background has been a huge help:
* Linear algebra: stuff like the QR decomposition for solving the "normal equation" of least squares regression, eigenvectors for PCA and the singular value decomposition for T-SVD, and so on. Linear algebra shows up everywhere in applied mathematics, statistical modeling, and data science. It's actually relatively simple and can be understood completely in about two semesters, but physicists do get a lot of practical experience and intuition. For example, eigenvalues turn out to be very important in quantum mechanics, so I suspect physics students spent a lot more time thinking about them compared to almost any other major.
* Vector calculus, matrix calculus, and optimization: physicists see this stuff in classical mechanics, E&M, etc. We can easily visualize vector fields and know a ton of relevant theorems and notation. We learn specific techniques such as the method of Lagrange multipliers for solving constrained systems. All of this helps because a huge chunk of statistical modeling and machine learning is formulated as an optimization problem, often a constrained optimization problem. For a statistician studying, say, SVMs for the first time, things like Lagrange multipliers and KKT conditions seem to come out of nowhere, while a physicist would have seen them in several other contexts first.
* Scientific programming: Most physics undergrads will have at least some experience with numerical optimization or simulation. This is a little different than application development or implementing an algorithm. The main trick is to be able to translate equations into performant, vectorized implementations. You also need to understand lots of practical things like rate of convergence, condition number, singular gradients, etc., just to be able to debug when things aren't working correctly.
* Experimentation: machine learning is basically empirical. Cross validation is essentially designing, conducting, and analyzing experiments. We improve our models not by proving theorems, but by forming and testing hypotheses. Basic stuff but extremely helpful in practice.
You have to remember that 10 years ago you couldn't really get a degree in machine learning or data science. (I believe degree programs and specializations with those names now exist and becoming fairly common, although I've had mixed results when interviewing graduates of such programs.) I would say the physics joins statistics and applied mathematics on the short list of degrees that, purely by chance, covered most of the relevant material. Of these, physics is probably the worst choice if you know you're going in that direction, because a physics degree only provides a thorough grounding in the prerequisites and little to none of the details. I had to learn much of that on my own through self-study.