Earlier quoted context omitted.
It's perfectly OK if that company actually operates in that country. It is not OK if the only presence they have is a mailbox and the actual act of operation is done somewhere else. A taxation race to the bottom is not what the fundamental principles of the EU are about. At best they are an unwanted side-effect.
No you are missing the point of the EU. No company should be forced to base in a country to sell there. Apple pays huge amounts of taxes in Germany despite not being based there. It just doesn’t pay much “income tax” because it’s not based there.
For example, I am customer of a German branch of a French bank. The company is based in Paris and all contracts clearly state that. The actual service is provided from a German office building. That's where almost all employees responsible for the German customers are located and where you sent letters to. That company pays "income" tax according to German rules to the German government for all services provided from that building even though it's based in France. (Surely they do some tax avoidance as well but this doesn't change the point.) Thanks to the EU freedoms they could just open up an office in Germany, employ people there, and provide services.
I am also a customer at a Dutch bank based in Amsterdam and that's where the service is provided from. They don't have a presence in Germany and don't pay German taxes. Thanks to the EU freedoms they can provide services to me across the border.
(For the record: Apple does have legal entities in Germany. For every transaction they just happen to use the most tax-efficient entity of theirs without much consideration of the real-life transaction.)