Latin is beautiful, but its purity and regularity may be overstated because of its prestige.
There are irregular verbs, sometimes with complete suppletive replacement of principal parts by what used to be other verbs (e.g. sum, esse, fui, futurus; fero, ferre, tuli, latum). There are verbs that use passive forms with active meaning (deponents) or perfect forms with present meaning (defectives).
There are arguably completely missing forms in the verbal inflection system (the Romans knew that some forms plausibly "should" exist, especially based on a Greek grammatical model, but simply didn't have them!).
There is sometimes unpredictability in which noun case should be used with a particular verb.
The noun declensions are apparently based on two different sets of Indo-European noun inflection paradigms, so nouns with similar nominative forms can end up being declined very differently.
There are ambiguities where different noun forms coincide, which can even create parsing ambiguities in literature (like confusion between ablatives and datives, many of which look identical).
The extent to which the perfect stem of a verb can be predicted from the present is limited, as sometimes stem reduplication is used, but sometimes just suffixation of something like -vi.
There are loanwords, even classically, from Etruscan, Greek, and to a lesser extent other Mediterranean languages (just thinking of that "hodgepodge" issue).
The meanings of purpose clauses with the verbs of fearing are arguably backwards from the English point of view (although I think the Latin version does make plenty of sense).
Native and nonnative speakers couldn't easily agree in antiquity about whether vowel length should be contrastive and (I think) whether consonant aspiration was phonemic. I guess the native speakers' opinion should matter more, except there promptly became such huge numbers of non-native speakers that they started to have a really humongous influence on the language.
There are spelling changes even within the classical period, so there isn't quite one single classical Latin orthography.
I guess there are many fewer irregular verbs overall compared to Germanic languages (which historically have had up to hundreds of at least partly irregular verbs). But if we want to count unpredictability of Latin perfect stems (which is somewhat akin to the main source of irregularity in the Germanic verbs: stem changes) as a kind of irregularity, Latin will also have quite a lot of these.