Earlier quoted context omitted.
Do you own a car? Commonly when speaking with a mechanic they'll ask questions like Driver's side or Passenger's side to get away from left/right. People very often describe things with just _left_ or just _right_. They don't say things like when I'm sitting in the car the door to my left is hard to open. They just say the left door is broken even when both them and the mechanic are facing the car head on. By having…
You aren't wrong. In that mistakes can be made. But if I ask you about your left hand, is that ambiguous? By your definition here, that is an ambiguous question. But, that just feels wrong. Edit: I'll note that it is amusing to compare to cars. It seems the etymology of the terms is literally "driver side".
It is certainly clear if one would say "left facing stern" or "left facing aft", but that's a mouthful when you can just shorten it (and the reference facing direction is not relevant). Bonus points if the shortened version can't be mistaken for another direction...
BTW, I'm 100% down for introducing dedicated words for "my left", "your left" etc vs just "on the left". It would certainly save me a bit of time when my family asks me to look for something and they flip between the two meanings in the same sentence.