Earlier quoted context omitted.
On the contrary, we do have a choice! We could use computable numbers instead of reals. A computable number is any number which is output by some Turing machine, or, equivalently, any number which can be found by some algorithm. e and pi are computable. You are right that Riemann integrals won't exist, but if you modify definitions somewhat, derivatives and integrals can be defined just as easily for computable numbe…
Numbers that can be computed by Turing Machines are countable, so list them: c_0, c_1, c_2, etc. Take any interval [a0,b0] (with a0 and b0 computable), and cross out your computable numbers until you find one, say, c_i, in the interval (including an endpoint). Take some strict sub-interval [a1,b1] (again with computable endpoints) within [a0,b0] such that a0 not in [a1,b1]. Now continue crossing off computable number…
But more concretely: no, the computable numbers aren't complete. Why would this make analysis messy? I didn't find the presentation and results in Computable Analysis (the book) all that different from what you find in an ordinary analysis course. Is there something it makes hard which I'm missing? (This might be impossible to answer without specific knowledge of Computable Analysis; remember that the definitions of, e.g. limits, are different.)