I have a similar system. My domain is catch-all, but I give everyone a unique email address with a bit at the end of the alias to indicate what my email rules should do.
For example, if I get an email at anything_s@mydomain.com, that will go directly to spam. I use this for everything from Google to every small website I sign up on. They usually only spam anyways. And I check my spam every now and then for if there's anything important - there has never been.
I consider whatever most normal businesses send me spam as well, as I don't care for most of it. Uber Eats, for example, sends a number of emails per each order. That is just spam in my eyes. If I'll use a service I care about, I'll give it an email with a different alias suffix that will never go to spam. But I almost never do.
This has keep out the phishing spam when websites leak my email address just as well as the regular "important information about a minor interaction you did with us" spam that comes from most websites.