When you think about it, it would actually be crazier if our common words matched exactly with biological evolution.
This is one of the cool things about math, there are a lot of aspects of math which stem from thinking, "well what if I took a square root of negative one?" or "what if geometry worked this way?" and then someone does the thought exercise and fleshes it out into something interesting. And then there just happen to be aspects of reality that end up being very nicely described by these what if questions
For what it's worth, you can avoid "imaginary" numbers entirely if you want -- you can do arithmetic on real polynomials modulo x^2+1, or equivalently on 2x2 real matrices of a special form.