> Does the research not stand on its own?
Academic mathematician here. It does, sort of, but .....
Say a mathematics postdoc is applying for a job, and she works in underwater basketweaving. The two underwater basketweavers in her dream department read her application and can see that she's brilliant. She's developed a new methodology for weaving rattan under 100 feet of ocean without even getting it wet. She has letters from the most preeminent underwater basketweavers in her field. She should be a shoo-in to get the job.
One problem though. A majority of the hiring committee works in different areas of research. One thinks that underwater basketweaving is overhyped. Another thinks that great work was done in the eighties, but all the interesting questions have been solved already. And these people don't put much stock in the letter writers: they have a reputation for writing glowing letters even for mediocre candidates.
And, so far, the hiring meeting is deadlocked until the basketweavers pipe in: Our candidate has published in Journal für die reine und angewandte Mathematik and Inventiones Mathematicae and Publications Mathématiques de l'IHÉS.
And now, all of a sudden, that postdoc is getting the interview, because her competitors have only published in Mathematische Annalen and Indagationes Mathematicae. (These are all real names of math journals.)
That was a bit long winded, and please forgive me if it's a bit silly, but the point is that we need mechanisms to compare researchers working in different subdisciplines, for researchers to "prove" to the community at large that they're doing good work. I wish we could eschew these games, and I certainly wish we could ditch the commercial publishing industry, but we compete with each other for jobs, for grant funding, and for fame, and we need to establish some kind of metric for evaluating researchers which is as fair as possible.
It is not perfect; certainly it is not completely fair. But I believe it works reasonably well.