Paid fast lanes are a bad idea, but I think 100% equality for all packets is just as bad or worse. Real time traffic should be prioritized over data transfers. Right now my video call gets clobbered when I download a file. It doesn’t have to be like that.
How should your ISP know? Diffserv might help, but I doubt some shitty application will set them.
You can either do traffic pattern analysis or just base it on what IPs they are connecting to.