A great lesson one learns as a physicist is that one must develop a new intuition mainly through years of practice. So when you say "It's very frustrating", I interpret it as "this doesn't seem reasonable to me". But it doesn't have to be reasonable - none of us have any intuition for what happens at scales far different from our everyday lives.
The real question you raise is a very good one - how seriously should physicists take mathematical theories. If we were building a statistical model of, say, house prices and construct a reasonable linear regression model, we certainly don't believe that the market plugs the parameters of a house into the model to decide the price. The model is an approximation of the real dynamics of the market and this approximation might not hold in the future.
On the physics front, I would argue no one would consider a quadratic in speed air resistance term in Newton's second law, a fundamental feature of the universe. One can build a reasonable model that results in that term and it might even be a good approximation for some fluids in some speed/density range.
But, when it comes to more fundamental (as of today) theories like quantum electrodynamics, electroweak theory, quantum chromodynamics (all quantum field theories), or even general relativity (modulo discussions of quantum gravity) - both the predictive power and accuracy of these theories is so stunning (matching all the data generated at colliders like the LHC), that one starts wondering if we are no longer dealing with models but a true description of nature. The mathematical descriptions are also so constrained unlike the house price example above, that one can't just make modifications to the theories without violating core principles (and experimental data) like unitarity, causality, locality, Lorentz invariance etc. This only reinforces this view that perhaps this is close to a true description of what we see.
Now it is entirely possible (but IMO not probable) that this whole view will be upended and replaced by a very different physical picture. In a sense, string theory (which is now discredited heavily in the public's eye but that's a story for another day) was an attempt at a different physical picture that resulted in very rich structures that had nothing to do with physical reality.
So, physicists say that because the more time you spent understanding and studying quantum field theory and as more experiments are done (all the collisions at the LHC verify the standard model's predictions including the Higgs once its mass was known), it only reinforces that there's something deep about the current theories even though we have several unsolved problems (dark matter, dark energy, quantum gravity, fine-tuning problems).
Addendum 1: I'll add a book that is not accessible to non-physicists but gives a glimpse into the actual struggle of research and building intuition for something very abstract:
https://www.amazon.com/Feynman-Lectures-Gravitation-Richard-...
Feynman, like many others, spent considerable time applying all his powers to understand general relativity from a QFT perspective but eventually it didn't pan out (for anyone).