Nice points. Okay, I revise my argument! What will be left are genes, in the society, in both men and women as required, that make the people, men and women as required, really WANT to have babies! Then back to my argument: The population will shrink on the way there and then will grow again.
There is another point on my side: It is commonly accepted that what children do is a mixture of both 'nature' (genes) and 'nurture' (that is, what they learned from their parents). There are common, strong suggestions that a daughter of a women who is really happy being a mommy is more likely to want to be a mommy. So, net, for a woman to WANT to have children can have come down from nature or nurture.
In my experience, some women REALLY want to have children, and some women REALLY do NOT want to have children. When I was in the ninth grade, I met and dated a girl in the seventh grade, natural blond, great figure, very sweet, who REALLY wanted to have children, knew it, admitted it. E.g., each time she saw the face of a baby with its common characteristics, she wanted to have a baby. Now, I'm not saying that she wanted to have a baby there in the seventh grade, but she wanted to be a MOMMY eventually. And she was, eventually.
And I've known women who had children or not and, in both cases, looked at motherhood as "giving up the best years of their lives and their careers doing low grade, menial scut work to take care of some MAN'S [bitter resentment and anger about the man] children". Sure, the position was partly just negotiations as "don't throw me into that briar patch" but also was meaningfully genuine.
Net, now, some women WANT to be mommies, and some don't. Does this situation inherit, via nature and/or nurture? My observations say that it does. My observations are not very good data, but if the observations are roughly correct then we conclude that the effect, even if weak, could be how Mother Nature will "find a way".
More generally I have confidence that Mother Nature will 'win' this one, will meet the challenge, will "find a way"!
While I believe that the 'way' I mentioned, that women WANT to have children, and that this will get passed from mother to daughter, is the one Mother Nature will select, there can be other ways. E.g., a society can get rid of anything like US Social Security and, thus, tell people in stark terms that they need to have children as a source of support in old age. A society can tell girls that it does not support single women or single mothers and, net, girls should look for good husbands or hope that they have good brothers or have good inheritance from their fathers.. There can be a role for some traditional religions that strongly emphasize 'traditional family values' and having children, whether the women want to or not. Some people might say that the result will not be a change in the gene pool but in the popularity of some religious, social, and/or political values.
Or one might just say that the 'Darwinian' effect will apply to whole societies, e.g., Finland: Finland beat Sweden, Russia, and Germany in wars but is losing out to the 'feminists'! Whatever the politics, at 1.5 children per woman, Finland is on the way out of it as surely as some genocide. So, maybe it's countries that will die off instead of people with the wrong genes. Still, I trust the influence of genes more than that of politics.
Your issue of "N generations on" is interesting: In part my analysis is saying that the women were much the same 40,000 years ago, which would be about N = 2000 generations! So, it would seem that we could extrapolate a few generations from the N = 2000 generations! But, in part my analysis is saying that, if you will, the current, big 'environmenal' changes, i.e., from women having jobs, will bring the biggest changes in the gene pool in 40,000 years and bring the changes very quickly, in, say, just a few generations, say, just 100 years. My reading of evolution is that, when the 'enviroment' changes, then genes can change very quickly. E.g., the genes in some of those weak, sick, or dead limbs on the tree are already gone forever.