Huh. It's like he's saying (because of libraries in the web browser) that we should be benchmarking HTML5 the way that we would a C64 or an Amiga 500... not as a general purpose machine that can implement arbitrary algorithms, but as a crummy machine where all the magic is done in hardware (or, here, libraries).
Part of me wants to resist that frame. The whole point of browsers running (sandbox safe) arbitrary code is getting the benefits of arbitrary code, right? Who wants to distort their problem solving to conform to the ad hoc details of the current state of browser vendor libraries? Isn't that what we're trying to get away from?
But then a part of me thinks... we do that with GPUs too, right? If you benchmarked a modern game that didn't use the GPU (and didn't distort what work was done to accommodate the particularities of the how the GPU expects computation to happen), its performance would be relatively speaking even more dreadful. Just like a web app that doesn't lean heavily on the browsers' libraries.
Obviously there is a difference in that browser constraints are mostly social (and thus kind of arbitrary), and GPU constraints (and those of the C64, for that matter) seem to be responses to certain physical realities.
I guess in the short run, he has the right attitude if you want to make code that actually performs well today, but wrong if you think that putting political pressure on standards bodies and browser vendors by offering up critiques like the original article is likely to improve things.