> Is there real harm in appreciating the beauty of the people around you (if that's indeed all you are doing)?
This is probably not harmful, but isn't really what people talk about when they say "objectification" (though I suppose you can describe it as "internalized objectification" if it involves your internal perspective of the targets being reduced primarily to what they provide you.)
Most of what people talk about is externalized objectification, which is outward actions which treat people as primarily to objects of your urges. And largely on the basis that this does cause harm to others, either directly (when specifically targeted) or less directly by a number of mechanisms (including normalizing devaluing attitudes toward the targeted class of persons) when less individually targeted but targeting a class.
> The term has become so loaded that it's hard to discuss without caveats and justifications, because a good many will argue based on the assumption that objectification is bad, rather than any actual harm.
You seem to believe that utilitarianism is the correct, objective, and universally accepted basis for moral discussions, such that discussing something as problematic for any reason other than harm it does is itself a problem.
(And you seem to describe it as harmful a priori, without pointing to a harm to anyone deriving from it, which seems problematic under its own standards.)