Why is everything based on likelihoods even though likelihoods are so small?

Because it works well in practice. And to elaborate, usually when something works well in practice it's because it has multiple desirable properties - the one you "ask for", but also other ones you get for free.

In this case maximum likelihood approximate bayesian estimation with a mostly reasonable prior. Furthermore you could look at the convergence properties which are good.

You could probably design some degenerate probability distribution that ml-estimation behaves really badly for, but those are not common in practice.

Working on nlme models for work these days - it does become a bit of a headache when asking “how much better” the model with -2LL=8000 is from the model with -2LL=7995 obviously one is better “more likely given the data” but what if the better one used 2 more parameters and is hence more complex and might be overfitting the dataset? Well then there are all these “heuristics” to look at, AIC, BIC, some sort of trick with a chi^2 distribution function- these are all just ways to penalize the objective function based on the # of parameters but it’s somewhat debatable which one to apply when and I have read that some parameter estimation softwares don’t even compute these values in the exact same way - I am not a statistician by training I just apply “industry standard practices” in as reasonably intuitive a way as I can but my impression has always been that if you wander far enough into the weeds you’ll find that stats often becomes a debate between many approximations with different sorts of tradeoffs and much of this is smoothed over by the fancy scientific software packages that get used by non-statistics-researchers one of the most frustrating parts of my job is reproducing SAS output (an extensively used statistics product) using free R language tools since a SAS license costs more than some sports cars… But what is SAS actually doing? And it’s never just taking a mean or pooling variance in the standard way you’d read about in an intermediate stats textbook it’s always doing some slight adjustment based on this or that approximation or heuristic

This tangent may have been unrelated or irrelevant but I’ve long concluded that in practice statistics is far less solved than people might expect if they’ve never had to reproduce any of the numbers given to them by statistical analysis

Another thing to note is that you're multiplying probabilities together. Since each probability is between 0 and 1, youre always shrinking the likelihood with each new data point. When you're doing this kind of analysis, the question you're asking is "given a model with these parameters, what's the probability I get exactly this sample?" Which, when you phrase it that way, it becomes more apparent why the likelihood is so small.

[dead]