You're creating a false dichotomy since there's always the obvious option of sustainability which is a fertility rate of around 2.1. And declining populations will likely create a far less sustainable world than increasing would. I described in a peer thread how fertility rates directly lead to the age ratios in society. Low fertility rates means you're going to have a low number of working age people for a large number of elderly people. On top of this, positive fertility rates mean that economies naturally increase in size even if you do absolutely nothing - negative growth rates mean that economies will naturally shrink.
We're nowhere even remotely near our production capacity for food, water, or energy - but we're already facing labor issues in achieving such. And in a low-growth future, which is our future, these labor issues will magnify exponentially. And it's not like you can just pay people more and everybody lives happily ever after - there simply won't be enough people to do everything that needs to be done, even if everybody's willing to work themselves to death. The expectations for what a society can provide will begin to unravel.
And I see no reason not to go sci-fi here because there's every reason to believe it's our future. Earth will not be humanity's only domain. It's practically inconceivable that we won't have thriving off-planet civilizations within a century and realistically - far sooner. And a century of unsustainable growth would be vastly less damaging to society than a century of unsustainable decline.