Earlier quoted context omitted.
But HTML never was “an SGML application” in practice, and I highly doubt it was ever actually envisioned as that. There may have been some tools out there that processed HTML as SGML, but none of the ones I know of did (most notably browsers). And in fact, in practice you could just use your own custom elements without worrying about validity and it’d mostly just work. This wasn’t even particularly rare. (There was t…
HTML the markup language was clearly intended as an SGML vocabulary - TBL himself said as much [1] and HTML also reused element names from the SGML spec/handbook as example/folklore vocabulary such as for paragraphs and headings. What browsers made out of it isn't the matter here, but even if it were, the "practical, real-world HTML out there" argument is mostly used to pull up the ladder by an ad company/browser car…
You seem to always start with the assumption that SGML was (and perhaps is) an end goal. I deny this.
HTML was designed as an SGML vocabulary, but, where it mattered, never implemented as an SGML vocabulary. If Tim Berners-Lee ever even expected it to be treated as SGML very much, I suspect he hadn’t thought things through well enough (though that could also just be hindsight bias on my part).
There has never been any particular virtue in HTML being an SGML vocabulary. No one that mattered (which mainly means browsers) cared about SGML, then or now, and no web developers or end users care about SGML, so being SGML is just needless complication and potential for confusion (due to that implying different behaviour from reality). SGML is a hideous, complex beast that no one wants to work with, and which almost everyone that has heard of it is glad is dead.
Yes, SGML had some nice ideas. Yes, we keep on reinventing parts of it. Yes, a variant of Greenspun’s tenth rule applies. But SGML was just too flexible/generic, large and ugly. It doesn’t actually solve things. And the current HTML parser is the best thing since sliced bread and my favourite popular file type spec by a large margin despite its size, because it’s clear, unambiguous, and implementable.