When I did analysis of hundreds of millions of predictions to analyze our prediction algorithm for trucks, I had a couple of metrics I would look at: MAE: mean absolute error: easy to understand metric. Given a stable system, this was my go-to standard. Bias: you put this one in. Generally, you want your predictions to be slightly pessimistic. RMSE: root mean square error. Most important one. Being 5 minutes off cons…
This is essentially why I use waze: When there is an incident or other major block of a highway, the delays can easily be hours. With waze, I am routed around them. This have saved me catching a flight and arriving to a meeting in time on several occasions, as waze routing essentially is minimizing RMSE. Being 5% late eats into my safety margin, but when a trip takes 200% my estimated travel time it ruins my day.