Maximum likelihood estimation and loss functions

Let's say you have some guarantee that I'm using the same die each time and that each of the rolls are independent. We play the game ten times and 1 is rolled the first 9 out of 10 times, with a 5 being rolled on the 10th throw. Now, you know that there's a common loaded die that can be purchased that has a weight to skew the probabilities and you further know that the loaded die rolls a 1 80% of the time and the remaining 20% spread evenly to the other values (so 4% for every other value).

Given a choice between the loaded die and the fair die, which is more likely?

The first model, call it $\theta_0$ is the fair die. The second model, with the unfair die, call it $\theta_1$.

The probability of the first model ($\theta_0$) is:

$p( 1,1,1,1,1,1,1,1,1, 5 ; \theta_0) = \frac{1}{6}^9 \cdot \frac{1}{6}$

(approximately .00000008269085843959)

The probability of the second model ($\theta_1$) is:

$p( 1,1,1,1,1,1,1,1,1, 5 ; \theta_1) = (0.8)^9 \cdot 0.2 \cdot \frac{1}{5}$

(or .00536870912000000000)

So we write a computer program to iterate through all the "models" to see which is the more likely. In this case, the iteration goes through two models.

The models can be Gaussians, with model parameters the mean and variance, say, or some other distribution with other parameters to choose from.

For some conditions on models and their parameterization, we might even be able to use more intricate methods that use calculus, gradient descent, etc. to find the MLE.

The MLE formalism is trying to say "given the observation, which parameters fit the best". It gets more complicated because we have to talk about which distributions we're allowing (which "model") and how we parameterize them. In the above, the models are simple, just assigning different probabilities to each of the outcomes of the die rolls and we only have a choice of two parameterizations.

I used to get by on, “it’s the parameters that make the data most likely”, like it says on the name. I think that’s what you are after.

Then I took a stats class, and I know to say, “the MLE is minimum variance within the class of asymptotically unbiased estimators” … that is, “efficient” and “asymptotically consistent“ in the jargon. (Subject to caveats.)

Then I took a Bayesian stats class and learned to say, “it’s minimum risk under a (improper) uniform prior.”

I also recall there is a general result showing that any estimator which makes the score function zero has good properties with respect to average loss. So zero’ing the score by maximizing likelihood is a good strategy. (If someone could remind me of specifics, that would be great.)

But perhaps Gauss had it right when he exploited the (known, but yet un-named) central limit theorem and used how easy it is to maximize the quadratic that sits atop its “e”. (https://arxiv.org/pdf/0804.2996, page 3, top). It’s so easy we had to find a justification for using it?

Perhaps a first course at grad level, but my engineering bachelors covered MLEs but we didn’t learn/use any of those formal language things. I think the core mathematics (and likely other pure science) cohorts were the only people who learnt it.