I'm not sure but it's likely both. I'm a physicist and google says there are about 10k in the US while there are 5.3k archeologists in the US, and physics is already niche amongst the physical scientists. Compare for example, there are 33k biologists, and I assume that's only pure researchers, not doctors, lab techs, etc.
edit: corrected a number. Also, I looked at "employed" meaning full time or part time research vs. just having a PhD in it. Like physics PhDs are 58K which sounds about right, just few actually get a faculty position, temporary (post-doc, staff scientist) or otherwise.
As for why I say both, the number of positions tends to also put downward pressure on number of applicants I think and makes it more likely people who study it or consider studying it consider other fields or jobs before even applying.