While I certainly have had a few annoying issues myself, full credit to the team behind DevOps (nee VSTS) for doing an amazing job over the last >12 months to overhaul and improve the product. I have been impressed at the pace things are improving. I have not had the same issues with build machines disappearing into the ether (we use Windows hosts however) but definitely agree that the (lack of) caching is maddening.…
I haven't had a build agent disappear, but lack of build caching is very frustrating. The hosted build agents are slow to begin with, let alone needing 5 minutes for every build to restore packages.
AFAIK there's no way to "only run changed tests" too.
EDIT: Forgot to add that, despite the above, I'm pretty happy with it. The whole VSTS experience (Wiki/code/PRs/etc) is pretty great, it's nice to have everything in a single place.