Earlier quoted context omitted.
Security is not binary. In this case it depends on whether disclosing your identity to the servers you connect to is a problem in your threat model. Saying "they are public so it's ok" is technical oversimplification.
It's not binary, but if your security depends on your public key being secret, something else must be wrong. It's the same as someone who depends on their IP addresses being secret. That's not something you can count on, and your security would be better served by designing your infrastructure with the assumption that it's totally public information.
The word public is just a name for a kind of key defined in the field of cryptography and doesn't necessary hold the same connotation in application protocols that use public key cryptography. One example would be ephemeral public key that would have to be kept private to be able to retain forward secrecy (although in such schemes DH is usually used).
Consider as well this situation that is closer to SSH. Suppose that you are MS dev working on NT kernel that by night also want to anonymously contribute to Linux. Obviously both camps would not like you to do that for fear of copyright infringement but OpenSSH shouldn't betray you anonymity that could be reasonably expected. It's imaginable that only one public crypto key pair would be needed to authenticate the server if password authentication is used for the client. The user doesn't expect the client software to silently generate key pair, much less that the same pair is used for every domain because the pair is not strictly needed. Although the user should inform himself how the software works a good software similarly shouldn't work in ways that are reasonably unexpected to people that are familiar with the domain. Ideally, if OpenSSH developers can't really foresee it working any other way OpenSSH should at least explicitly inform the user which public key will be used in connection before it is established in order to not assume the consent of poorly informed users.