While HKers are Chinese, they speak an entirely different language than Beijing that isn't even mutually intelligible.
This is part of the manipulation of the narrative by the Chinese government. It refuses to even recognize that these are truly different languages and calls them "varieties of Chinese" or "dialects of Chinese" when Cantonese or Taiwanese are more distant from Mandarin than the Romance languages are from one another. Then it tries to pretend like there are only a single people in China and is systematically eradicating minority communities.[1]
Thus, Mandarin political domination over other Chinese communities is imperialism at the very least, if not colonialism. Imperialism and colonialism aren't really that different from one another.[2]
Additionally, every group always like to claim the land that they had at their maximal imperial state. But borders change over time. People move around -- even Taiwan was only recently colonized by Chinese in the 17th/18th century in large numbers and before and after that was owned by other empires. Another example is when the Dutch decolonized Indonesia. The Dutch were simply replaced by the new "foreign imperialists" of the central Indonesian government, which the Balinese people and other such peoples of Indonesia considered a different group than themselves.
Furthermore, simply a region has been part of a broader empire for centuries or even millennia, doesn't mean isn't still being oppressed by an imperialist empire. The Russian Empire, Spanish oppression of other provinces and minorities for hundreds of years, etc.
Thus perhaps instead of focusing on the colonialism part we should focus on the imperialism part. The aim should be to empower self determination of a people and enabling them to be free of oppressive, imperial influence.
[1] https://www.bbc.com/news/world-asia-china-53220713
[2] https://plato.stanford.edu/entries/colonialism/