JWST, like many other spacecraft, has reaction wheels to orient itself. The reason Kepler ran out of fuel when it did was that it was expending more maneuvering fuel for attitude control because two of its four reaction wheels failed. Hubble and the ISS also had similar failures.
We're getting better all the time at building more reliable components (including reaction wheels and cryocoolers) though. Until a few years ago, the life of something like JWST would be limited by the amount of liquid helium on board to cool the components. Modern cryocooler technology (aka a space grade refrigerator) is good enough to cool it indefinitely. Solid state cryocoolers, previously unachievable, are now apparently available for some applications (important not only for reliability but also to reduce vibrations).
Reaction wheels can be used for attitude control but they still have to be unloaded by thrusters after maneuvering for a while. You're right that you could use a rudder (probably two rudders would be required for 3d attitude control) and have to have a balanced solar wind profile (JWST does actually have a solar wind balancing flap, but I don't think it's adjustable like a rudder). But solar wind won't act fast enough if you want to quickly change attitude for observations. And you can't use reaction wheels for stationkeeping. It very much matters where the telescope is, since if it drifts too far away from Earth it will be much harder to send high bandwidth data, and if it's too close to Earth, Moon etc. it will have no way to orient without heating up or blinding itself with the IR sunlight reflected by them.