Floats Are Weird

This is an interesting question to answer in a way that works regardless of mathematical background; let me try something a bit handwavy.

The way people are taught about the real numbers is typically a progression, you work with integers as a child, get your head around negative numbers etc., eventually you are shown "algebra" and equation solving, and you will be introduced to "roots" as solutions. You'll learn about e.g. sqrt(2) as the solution to 2=x^2 and probably have some discussion of irrational numbers then, rather than rationals, but it may miss most details. If you study in university at all you'll probably get some sort of lecture on countable vs. uncountable infinities and probably not look too closely unless it's a pretty mathematically oriented class.

Along the way you'll also be introduced to "special" numbers, pi and e at the minimum, usually motivated from somewhere else (e.g. we "found" pi due to geometry, e comes from logs, etc.).

So the picture you get is that you have all the "normal" every day numbers, then some more a little bit weird like sqrt(2), and a few special ones like "pi" and "e" that are useful.

Truth is, from the point of view of the reals, this is all backwards.

Numbers like 1,1/2, sqrt(2), etc. are called "algebraic" because you can find a polynomial equation with integer coefficients that has it as a solution (so x^2=2 means sqrt(2) is algebric, x=7 means 7 is, etc.). Anything non-algebric is called transcendental. Pi is transcendental, as is e, which means no matter how hard you try you can't find a polynomial of that type such that ax^n + bx^n-1 + ... = pi. It turns out to be hard to prove that something is transcendental, we have only proven a double handful or so.

So the weird part is transcendental numbers are "almost all" of them. In a technical sense: the set of transcendental numbers has measure 1 in the field (of real numbers,o r complex for that matter).

One way to think about it is if I gave you a bucket containing the real numbers between [0,100] and you randomly picked numbers out for the rest of your life, you would expect to never pick an algebraic number.

So all the numbers all humans use for day to day math and accounting, as well as most scientific math, etc. (obvious pi, e and friends contribute) ... all those number come from a subset of the reals so small as to be negligible in the grand scheme of things. To a first approximation, the reals consist of numbers nobody ever uses :)

> when more reasonable modern default choices for all purposes outside of high performance applications would either be a rational composed of bigints, or a bigdecimal, or maybe a generating function for the coefficients of a Taylor series expansion or something like that if you really want to get nuts.

Rationals of bigints doesn't work in general. First of all, you still have to do approximations if you take e.g. a square root or a sine. And even if you have an algorithm that is closed under rational numbers (like the Simplex method for solving linear programs), the number of digits you need in your integers blows up unless you have a tiny problem.

My rough sketch is that inv(A) involves det(A) in the denominator, and the number of digits in det(A) is exponential in nature (n! if A is n-by-n).

Kind of OT: Why do so many people say “phenomena” when they want to use the singular case?

Well, not to be too glib, but I don't think they're too much stranger than I realize, and I think floats are much harder to reason about.

How floats work is a non-trivial part of my 3rd semester CS course (Computer Architecture) in the US, I'd imagine the same applies to every other good college.