Earlier quoted context omitted.
Why do people say '2 is a solitary outlier'? Sure, by a quirk of English there is a special word 'even' denoting 'number divisible by 2'. By that logic, every prime k is a 'solitary outlier', being the only prime divisible by k. E.g. '5 is a solitary outlier, the only fiven prime'.
And while we are at it why is 1 not a prime? It conforms to the definition.
Why would they choose a definition so that 1 is not a prime? They found it the best and most useful definition that they knew at that time.
While there are many bad definitions, concepts and names in math, I don't think prime number is an example.
Also please notice that sometimes the more useful concepts have longer definitions, especially if one limits themselves in the terms they want to define it. (That is they don't use the nicest properties to define prime numbers, but they define them as:)
A prime number (or a prime) is a natural number greater than 1 that is not a product of two smaller natural numbers. [0]
This is indeed longer and more complicated than if we did not exclude 1. But the concept is (at least in the current state of math) more useful than if 1 weren't excluded. Who knows, maybe in the future math they will find a new structure or theory, where the prime numbers + {1} is a really useful concept.