Your argument assumes that the world of "stuff" resembles GNU/Linux. Very broadly useful things that lots of people have the expertise to contribute to, where the costs of contribution are low.
Most stuff doesn't look like that. Nobody is designing transmission power control loops in their free time, nor are they designing chemical processes to remove impurities from natural gas before combustion. You gotta pay someone to do that, and it's very expensive to do so.
Also, patents are often construed as a reward for innovation, but I don't think that's the only way to look at them. I think a better way to look at them is like other abstract types of property, e.g. stock in a company. Patents, like stock, allow division of labor and specialization. Nobody is designing high-performance CPU cores in their spare time, but Samsung could pay someone to do it. Without patents, they'd keep the design a trade secret, and NVIDIA, Apple, etc, would pay someone to make their own designs and keep those as trade secrets too. Patents (and copyright), allow a single company, ARM, to specialize in designing CPU cores, by creating property which can be the subject of transactions between ARM, Apple, Samsung, etc. In such a scenario, the patent isn't a "reward" for innovation, but simply a legal abstraction that makes certain types of business arrangements practical.