More accurately, you have spotted not a Linux user in general, but a user of certain Linux distributions, which in my opinion have inadequate display configuration settings.
I am also using only Linux on all my desktops and laptops, and I have never used any display with a resolution less than 4k, for at least the last 12 or 13 years.
Despite of that, I have never encountered any problems with "scaling", because in Linux I have never used any kind of "scaling" (unlike in Windows, which has a font "scaling").
In the kind of Linux that I have been using, I only set an appropriate dots-per-inch value for the monitor, which means that there is no "scaling", which would reduce graphic quality, but all programs render the fonts and other graphic elements at an appropriate size and using in the right way the display resolution.
I configure dots-per-inch values that do not match the actual dpi values of the monitors, but values that ensure that the on-screen size is slightly larger than the on-paper size, because I stay at a greater distance from the monitor than I would keep a paper or a book in my hand (i.e. I set higher dpi values than the real ones, so that any rendering program will believe that the screen is smaller than in reality, so it will render e.g. a 12 point font at a slightly bigger size than 12 points and e.g. an A4 page will be bigger on screen than an A4 sheet of paper; for instance I use 216 dpi for a 27 inch 4k Dell UltraSharp monitor).