They did the same thing to our messaging app where I work. Claimed we were uploading contacts and removed our app; we definitely don't upload contacts, that's a fact; the infra is also self-hosted so where would we be uploading them to? They only way we got them to allow it back was to add a privacy-policy notice to say that we upload contacts and why, despite the fact we actually don't...
...? How does self-hosted make a difference in this scenario? Uploading means it's leaving the phone, regardless of the destination. It could be going to your server or AWS or anywhere else, and I would consider it being "uploaded".