HTTP/2 server push is odd. When you can get the most benefit out of it, you usually don't care, because your resources are small and load quickly anyway. And when you care (because your resources are big don't load quickly enough), HTTP push actually hurts you, because you are pushing the same big resource on every page load. I have tried to use it once, and the hassle of distinguishing between first time visits and…
Size is only semi-related to latency. For small resources, latency costs dominate. That's what push addresses.