Is that unreasonable? Running another OS on a device with physical hardware switches is a privilege right now, so it costs more. There are essentially two companies doing this, and neither of them are even at the point where they can completely honestly say their products are out of beta. The Librem is expensive in no small part because its feature list is fringe, and even ignoring the inherent hardware challenges that's just how the law of supply and demand works.
What other market have you seen that acts differently? Look at how much other niche devices like Braille screens cost, or even more "mainstream" specialized hardware like the Remarkable Tablet -- arguably a much less capable E-Reader than a Kindle/Kobo device by almost all consumer-relevant metrics. But you'll pay more for stylus support on an E-ink screen if you need that, and you'll tolerate a suboptimal reading experience as well.
It's also important not to ignore the fact that the Librem is a flagship device. It is currently the most powerful Linux phone hardware on the market. So you can buy a cheap, low-powered Pinephone (the far better choice for most tinkerers), or you can shell out for something with more raw power -- which is also pretty consistent with how most markets work.
If you want to buy the single top-of-the-line Wacom tablet, you'll shell out at least $4,000, probably more when you factor in accessories. You want rotational support in the pen? That's another $100. Is the fractional improvement in a Cintiq Pro 'worth' the frankly massive price increase over consumer tablets? Probably not, you can buy an entire Surface Studio for the same price, and that comes with a "free" computer. But you'll pay the Cintiq price if you belong to a niche that needs the best the market currently has to offer for your particular use-case. And you'll pay the Librem price if you belong to a niche that needs the (currently) most powerful phone hardware that's realistically usable with a Linux OS.