You're right, we don't want that sort of adoption! We want slow, paced, organic growth. Users should join the fediverse because they want to be there, not because a giant corporation cut their accounts over to a new service that supports some random thing they've never heard of and don't understand. The culture clash that will result from Facebooks decision could be devastating, so naturally people are pushing back against that kind of adoption.
> with the smaller ones making themselves irrelevant if they do
I don't think this is true. The fediverse started out as a way to specifically have small services where users (mostly trans and queer users early on) wouldn't be harassed all day. "Irrelevant" only makes sense if you're thinking from the perspective of "get as many users as possible", but the fediverse isn't really about that. Small servers of marginalized people talking in private spaces where no one is harassing them is still extremely relevant.