I think the key is not to try to avoid biases (which sounds like strange advice, I admit). Discrimination is how to separate bad from good. Everyone also has preferences which help us select good (for us) results. We have experience which guides us to choices that have been successful in the past. All of that is bias -- it influences our choices and not all of it is bad.
It isn't bias, per se, that is problematic. What creates difficulty are the consequences for poor choices. What we colloquially call "discrimination" is really someone being blind (or uncaring) to the results of their own poor judgement. All poor judgement is harmful, but there are some classes that we single out for special treatment in society.
There are many examples in society where we make "allowances" based on discrimination. We might accept smaller women fire fighters than we do men. We might accept people from certain religions to wear slightly different uniforms. We might allow people with a physical impairment to have an aide for taking a test. By and large, these are choices that are meant to create good consequences, but they are discriminatory nonetheless. Indeed, it is societal bias which determines which allowances are considered to have good consequences and which are not (I expect that some people would question or reject one or two of my points, for example).
Of course, not everyone is good at making judgements. Additionally we sometimes have cultural biases which encourage us to make certain poor judgements. It's important to realise that all poor judgement is harmful. Honestly, it doesn't matter if you make my life a misery because of my skin colour or because you don't like the way I talk. It doesn't even matter if you do it simply because you are unskillful and don't realise (or care) that you are doing it.
But you can't make it illegal to be a jerk. Everybody would be in jail at one time in their life. Because there are some classes of problems that are pervasive in society, we can address the problem there. We can address specific problems with large classes of sufferers. The other potential benefit is that hopefully people will start to realise that their behaviour is undesirable. So if they think, "I can't use sexual innuendo as a way to politically lock out a female rival", they might start realising that politically locking out any rival is not really a beneficial strategy.
I say might. I'm not really expecting that much, but one can live in hope ;-)
Sorry for the book. I feel frustrated in the same way that I imagine you do. I just think that we need to help people steer away from the "thought crime" of bias and start getting them to focus on the "real crime" of harassment. If we complain too much about "bias" (or the allegation of bias), I think we risk people getting stuck on that point and never getting around to discussing how we learn to act better to each other.