While Toyota has been busy looking for silver bullets that will solve imagined problems, Tesla has been scaling hard with good-enough technology. Their Q2'23 production is a hair under 2 million units per year, so they're already a mid-size OEM (the size of BMW, MB, Audi, etc).
Maybe Toyota does not want to compete in the luxury EV business because they do not see a profit for them. Toyota's success was not built on their luxury models.
Extra $8K on a $50K car isn't too bad and can be hidden by cutting costs somewhere else. But on a <$20K car there's nothing left to cut, and the extra cost takes it out of the budget segment.