"It does not have to be", but in practice it usually is (although he accepts BSD, even though it doesn't guarantee users' access to modifications). RMS advocates "liberated" software, or software that is itself free from being locked into a proprietary fork. That's the whole point of the GPL family of licenses. If you read the corpus of his works - I have - he very consistently talks about the importance of software freedom and how that Open Source can refer to many very non-Free things.
> Do people even read this whenever they quote it?
Did you? He writes:
"Yet some open source supporters have proposed “open source DRM” software. Their idea is that, by publishing the source code of programs designed to restrict your access to encrypted media and by allowing others to change it, they will produce more powerful and reliable software for restricting users like you. The software would then be delivered to you in devices that do not allow you to change it.
"This software might be open source and use the open source development model, but it won't be free software since it won't respect the freedom of the users that actually run it. If the open source development model succeeds in making this software more powerful and reliable for restricting you, that will make it even worse."