We do use electrons. That's what flows through transistors to do computations. Or, vaguely, the distribution of the electric field....
>We got pretty damn close in the vacuum tube era
Uh, what?
There's only so many fundamental interactions in the Universe. Computing requires you to be able to distinguish two states and our current methodology is built around some sort of black box three input machine that can output either state, a switch.
That switch is the part that cannot be scaled down infinitely. The reality we are familiar with doesn't exist at atomic scales. "Things" don't even have properly defined boundaries at a certain level, and thermal noise is a huge issue.
IMO a much more direct limiter of our current computing capability is lack of manufacturing ability, and heat. We were lucky that transistors were so amenable to lithography as a concept, that they work so well in 2D and as a surface feature, as that is what drove our advances the past 100 years and enabled computing to be such a normal thing. The combination of a "Solid state" effect, the electric force having very convenient properties, and lithography being so amenable to scaling things in various directions is how we got here.
But lithography doesn't scale into 3D. We've been hacking around that by doing more layers but that scales awfully, has very strict limitations, and makes the heat problem infinitely worse, to the point of making it impossible to work around.
If we could assemble things atom by atom exactly how we want, we could vastly improve our theory and practice, and build really intricate processor chunks with effective cooling channels or something, and computing would scale so much more. Maybe. Maybe some other problem would suddenly start dominating in that world.
Biology literally is nanotechnology, but it takes massive tradeoffs in exchange. It might never be possible to manufacture, at scale, stuff atom by atom. The Universe doesn't promise us infinite progress in technology. Quite the opposite.