I could understand how this applies to state universities, but how does the US Constitution have this kind of jurisdiction over Harvard?
Private college status doesn't override basic rights elucidated by the Constitution which are further spelled out by various civil rights laws.
I think the answer must be that Harvard has government contracts or something (as others responded to me).