Replace usage of "Toxic" by "makes me feel like shit" and I think you'll understand better.
What would you want to debate against someone who says something makes them feel like shit? There's not a lot to say beyond: "I don't care how you feel", "That sucks you feel that way, what would make you feel better", and "It's unreasonable to ask me to change in ways that make you feel better, so suck it up".
Now imagine one person telling another their behavior is toxic, which means they are telling them that they make them feel like shit. Well the response will dictate what happens to the relationship. If the response is anything but "That sucks you feel that way, what would make you feel better", well the logical move for the person feeling like shit is to find a way to never have to interact with the toxic person again or force them to change their behavior.
What else would you want to discuss?