I've got a ~80k library and it seems to handle it ok. Scanning isn't very fast, but it does pull up a lot of cover art and artist profile pictures, so that may be why. It doesn't seem to respond at all to changing tags for files it has already scanned, so far the only solution was removing the library entirely and re-adding it which is tedious.
It supports drilling down by Album Artists which is a minimum requirement for nay big and diverse music library. Genres seem to work fine, you can select one and it will at least list all albums under that, but no option to change the grouping to artists or something else exists.
The web interface does ok at first when loading a big playlist. For example, trying to play my OC-Remix collection which is essentially an album composed of ~4k songs works for the first few songs and moving around the playlists is somewhat smooth, but picking a song at 1000 place or so just awaits forever. I don't know whats up with that.
The playing queue is very terse and doesn't like listing the track number, artist nor even the album (it does show a thumb sized album art for each entry but good luck with that). This is a very dumbed down mobile centric design, could do much better here...
All in all I would advise just trying out more than one media server. Jellyfin is ok-ish for music and great for videos but if you are gonna go trough all the work of setting that up it doesn't cost much more to also setup something like Navidrome which will give you a decent subsonic instance which is compatible with many clients and will likely give you a better experience when handling big and diverse music libraries. That said, Navidrome's built-in web client is pretty bare-bones, so you are gonna have to rely on the "native" clients more.
For example, you can use Strawberry's subsonic feature to access your Navidrome's library on a pretty damn good interface for music nerds who feel comfortable on "spreadsheet" like interfaces. No tag editing, but you don't usually want to expose your library with write permission on these services anyway. But it does pick up on tag changes if you tell it to a complete re-scan which takes ~15 minutes.