I'm not saying that you shouldn't ensure that your website is crawlable by search engines. You absolutely should. But we're at a point today where you simply don't need to worry about whether your site works without Javascript or not for the functionality of the site. Even Mozilla is removing the option to disable Javascript from the browser's UI (you can still use the advanced about:config or extensions).
As mentioned elsewhere, screen readers fully support Javascript and have for a very long time (which you've updated your post to acknowledge). That was one of the big reasons I used to ensure sites worked without Javascript years ago.
NoScript users know enough to turn it on for a site for it to work. NoScript users only make up 0.08% of internet users worldwide, a far cry form 1-5%. Basically, you can safely give them a significantly reduced site experience or just a message to turn Javascript on. They're not really worth the effort in most cases. Hardly anyone disables JavaScript anymore as most sites simply won't work right without it.
If you want to expend the extra effort, more power to you. It's just that for most sites nowadays, it's not worth the time/money anymore.