I agree, but there is a lot of value in approaching Philosophy from a mathematical context (both as a student and as a philosopher- c.f. Russel, Wittgenstein, et al had backgrounds in mathematics and left a huge mark on the subject).
I was a math major for much of my undergraduate career, but by my junior year I realized that I was actually only interested in the philosophy behind it. In fact, I had little interest in my freshman Philosophy 101 course, though after studying math, I realized that I would have loved to study philosophy more. As a freshman, I was enamored by math as it was this ivory tower of abstract truth.
I loved learning about what math could say and how the notation worked and the notion of formal proof etc. but slogging through proofs was abhorrent after awhile and I dropped the program. I had come to understand that math was actually just the same sort of discourse as "soft" philosophy, just formalized. Learning about counterexamples to math as a "closed system of reasoning"[0] (Godel's theorems, constructive mathematics, etc.) simultaneously ruined my perception of the nobility of math and spurred my interest in philosophy (specifically analytic philosophy and philosophy of language) and cognitive science. That's where the real "unsolvable" or "interesting" problems lay (e.g. the mind-body problem).
I would highly recommend anyone who is intellectually curious to learn and understand formal mathematics for the purposes of understand philosophy, but also to recognize when to quit (if ever!).
[0] - I may be butchering these terms, but I hope the meaning is clear