MathML is a failed web standard • Hacker News Board

Re: MathML is a failed web standard

#161

Post by Mikhail_Edoshin » Fri, Apr 08, 2016, 10:18 AM UTC

bentley wrote: post #22

It’s instructive to see how different MathML is from other math markup languages. Here’s the quadratic formula: In troff, x = {-b +- sqrt { b sup 2 - 4ac}} over 2a In TeX, x = {-b \pm \sqrt{b^2-4ac}} \over {2a} In plain Unicode, 𝑥 = (−𝑏 ± √(𝑏² − 4𝑎𝑐))⁄2𝑎 In MathML, x = − b ± b 2 − 4ac 2a MathML is simply unreasonable to write by hand. Most of the time it’s only ever used as an interchange format, automatically…

XML is not meant to be written primarily by hand (although I agree some XML languages could be more elegant). The markup part of XML is for the machine; if you remove it from your example, you'll get x=−b±b2−4ac2a, which is, basically, the text content that was meant for the humans.

Now, as a machine language XML is much better than troff or TeX, because it's very easy to parse: it's basically a syntax tree, the result of parsing those other formats, you don't even really parse it, just deserialize. Naturally, it's a very good interchange format. Technically it would be a much better option than a full-fledged JavaScript parser and typesetter because it would've removed the parsing part. (And this is only one of the advantages.)

Re: MathML is a failed web standard

#162

Post by Mikhail_Edoshin » Fri, Apr 08, 2016, 10:57 AM UTC

koolba wrote: post #51

spicyj wrote: post #23

Earlier quoted context omitted.

Yeah. I think that's inevitable with an XML syntax. XML is good at some things, but representing math expressions clearly is not one of them.

> Yeah. I think that's inevitable with an XML syntax. XML is good at some things, but representing math expressions clearly is not one of them. What is XML good at? (by good I mean better than alternatives like JSON, YAML, HAML, etc) The only thing I that might qualify is a long term/archival quality document format like ODF/OOXML. The inherently embeddedable nature of XML does seem like a nice fit but it gets very b…

1. XML is very easy to parse (linear-time). By comparsion, TeX or troff, while elegant, are not that easy to parse. In the sample formula (see the first comment) the parser that reads the beginning of the formula has no idea it's going to end up with a fraction until it sees \over. So it's a real parser and then a post-processor that sets things up; check "TeX: The Program" for details. And it can only parse one language. In XML it's just a very dumb highly optimized generic loader that can load any XML. I agree the content has to come from somewhere, but it's a different story.

2. XML data model is more sophisticated than JSON or YAML: it supports element ordering and mixed content and does it rather elegantly and succinctly. It also has namespaces (and these are very good namespaces, they're not hierarchical, they're just long names in a single flat namespace with convenient notation to shorten the long prefixes to reasonable size). As a result it's very easy to define a new language, extend a language, mix multiple XML languages, etc. JSON and YAML are hopeless here.

3. XML comes with tools to define the type of the document or a fragment, so you can read a document and automatically check that it has the right syntax (and/or convert the data, such as dates, into the native format). There are three ways to do this (DTD, Schema, Relax NG) in order of increasing power and expressiveness (not just syntactic sugar, but different kinds of languages). In particular, it natively supports things like inter-element references, which is very convenient for complex documents.

4. XML comes with XSLT, which is a general-purpose tree transformer (transducer) with declarative syntax. This is an immensely valuable tool. To put things into perspective: a compiler is a special-purpose tree transformer that transforms the source tree of a program into machine code (which is also a tree, technically: sections, data, functions, etc.). Are you sure you don't need a general-purpose declarative tree transformer and prefer to write ad-hoc ones? :)

5. The specification of XML 1.0 is shorter than, say, YAML :) OK, this is only one part of XML landscape, the whole is much bigger, of course; but still this part (basic XML and DTD) is noticeably shorter than YAML. (I myself also find YAML pretty cryptic.)

Re: MathML is a failed web standard

#163

Post by mstade » Fri, Apr 08, 2016, 11:09 AM UTC

Mikhail_Edoshin wrote: post #162

koolba wrote: post #51

Earlier quoted context omitted.

> Yeah. I think that's inevitable with an XML syntax. XML is good at some things, but representing math expressions clearly is not one of them. What is XML good at? (by good I mean better than alternatives like JSON, YAML, HAML, etc) The only thing I that might qualify is a long term/archival quality document format like ODF/OOXML. The inherently embeddedable nature of XML does seem like a nice fit but it gets very b…

1. XML is very easy to parse (linear-time). By comparsion, TeX or troff, while elegant, are not that easy to parse. In the sample formula (see the first comment) the parser that reads the beginning of the formula has no idea it's going to end up with a fraction until it sees \over. So it's a real parser and then a post-processor that sets things up; check "TeX: The Program" for details. And it can only parse one lang…

Great summary! XML really isn't a bad format, it just gets bad mouthed by everyone who thinks it's a bitch to author -- and I agree, it is. But just don't author it by hand then. JSON or YAML of any reasonable length is also terrible to author by hand, yet you lose out on so many of the benefits of XML. And for what, better "hello world" samples?

Re: MathML is a failed web standard

#164

Post by ga6840 » Fri, Apr 08, 2016, 4:44 PM UTC

dginev wrote: post #160

ga6840 wrote: post #145

Earlier quoted context omitted.

Agree, we do not include a 4*4 image in HTML by inserting ...... In browser level, I think we should treat math expression as a simple and atom component, and the only benefits to expose DOM/XML/JSON or whatever structural information in webpage is probably you can manipulate/extract info from it (e.g. using Javascript). Do we really need to manipulate a math expression? I think a simple " \frac a b " makes much sens…

"Do we really need to manipulate a math expression?" If you don't, please don't say no one does. Having a math DOM allows for actual interactivity with mathematics, from highlighting, copying subtrees, embedding links, having on the fly computations/simplifications, etc. Not to mention add-on services like math indexing and search. Yes, we need to manipulate and machine-read math expressions, if we want to finally ta…

What I mean is "really need". In fact, there is also the possibility we want to highlight a portion of an image, copy a subimage, etc, but did our HTML tag designed like the way I mentioned? I am the author of a math search engine OPMES (tkhost.github.io/opmes), the search engine works pretty well without the knowledge of DOM structure of math expression. Actually MathML makes a lot inconvenient during OPMES development, to a degree that I choose not to support it.

BTW, if we want a tag that no one will write by hand and only machine will try to understand, then think about why not HTML being designed as some open binary format in the first place?

Re: MathML is a failed web standard

#165

Post by ga6840 » Fri, Apr 08, 2016, 6:00 PM UTC

dginev wrote: post #158

ga6840 wrote: post #143

I am building a project and doing research on math-aware search (my project is hosted on https://github.com/t-k-/the-day-after-tomorrow ) As for the search engine for math, it is a pity that MathML has become a standard "input" for mainstream research. The most famous conference on Math search: NTCIR, is actually publishing its main dataset/corpus in MathML. Converting MathML back into LaTeX is possible but error-pro…

I am the person behind generating the original NTCIR math datasets, and probably most of the research-produced MathML out there. We've recently presented that we have more than 350 million formulas from arXiv converted over to MathML, together with the rest of the papers as HTML5. As someone who has stared at arXiv TeX/LaTeX for years, I can testify you don't want to be looking at TeX math in actual latex documents,…

1. In NTCIR (main) dataset, I see many cases where does not contain an altext (and thus no TeX). I asked LaTeXML author Bruce Miller about this, he said LaTeXML will always put the same TeX string as an altext attribute on the . So I assume you guys are using some out-dated LaTeXML version? I really want to plead NTCIR to ensure the original LaTeX annotation is kept in main dataset, or please provide both MathML and LaTeX version corpus for researcher to freely choose. This will allow LaTeX-only math search engines being able to compare results with other MathML search engines. You know it is hard to convert all of them back into LaTeX correctly.

2. I wish NTCIR corpus is not that difficult to download (I once wrote a request for NTCIR corpus, but no one replies), please make it public accessible just like what MIaS does: https://mir.fi.muni.cz/mias/

3. My search engine (http://tkhost.github.io/opmes) is actually using structural method, but I still give up MathML and go parsing TeX directly instead. Why? In TeX I can just omit irrelevant command like "\color" and "\mbox", and only focus on a handful math-related TeX subset, and the result is great. Although my search engine can just handle "toy formula syntax", but maybe it is better than MathWebSearch (https://zbmath.org/formulae/) and even beat Tangent (http://saskatoon.cs.rit.edu/tangent/random) in long query. But in MathML, I have no idea why I need to read its lengthy spec, and I see no reason to write a MathML parser.

NTCIR-math conference (and its none-friendly website) makes me unwilling to submit a single paper.

Re: MathML is a failed web standard

#166

Post by Mikhail_Edoshin » Fri, Apr 08, 2016, 7:25 PM UTC

pcwalton wrote: post #104

hansen wrote: post #57

Earlier quoted context omitted.

This is about how I feel. Can someone who’s actually competent explain why (La)TeX output looks so much better than what web browsers do? I’m not talking about childish colors, sticky headers, and all the other annoying “modern web features”, just the relevant content: some text and inline images.

I don't think TeX output looks better than what Web browsers do if you enable all the high-quality text features like hyphenation and use modern OpenType fonts (in particular not the Core Fonts for the Web). The reason why the Web doesn't do this by default is backwards compatibility and performance: hyphenation is asymptotically more expensive.

TeX doesn't hyphenate every word, it only attempts hyphenation when it looks where to break a line. That said, I don't see why a document cannot be prehyphenated once on the server; I'd say this is the right thing to do. (Besides, in this case we don't have to rely solely on algorithms.)

It's not that CSS is still very far from professional typesetting; what amazes me is that at the same time it's much more complex. Look at all these possible 'display' values and box models! And you still cannot do a run-in header! Or automatic numeration! (You can do the latter natively with XSLT, but it's not cool.) By comparison TeX only has boxes, vertical and horizontal lists, glue, and a few lesser things like kerning. And it produces works of art.

Re: MathML is a failed web standard

#167

Post by Mikhail_Edoshin » Fri, Apr 08, 2016, 7:33 PM UTC

ealloc wrote: post #80

1ris wrote: post #48

Earlier quoted context omitted.

In a more liberal sense the opposite of "open standard".

I wouldn't describe TeX as a "standard", but it's pretty hard to argue that it's "closed": TeX is arguably the best documented program in history! Knuth wrote it using his novel "literate programming" technique in which the source code is embedded in the documentation. Every 4-5 lines of source code has roughly a paragraph of explanation. That source/documentation is published for anyone to read as Knuth's "TeX: The…

I wouldn't call it easy reading though :) By the way, it's written in Pascal, while all current implementations, afaik, are rewritten in C.

Also, I found "TeXBook" very hard to follow; "The Program" is much more understandable. Every chapter in "TeXBook" starts with a few paragraphs of relatively simply text and then descends into a bunch of additional paragraphs marked with "dangerous turn" signs that talk about things that were never mentioned before. As the author put it, they're explained "somewhere", but it's really hard to find that somewhere.

Re: MathML is a failed web standard

#168

Post by dginev » Fri, Apr 08, 2016, 10:54 PM UTC

ga6840 wrote: post #164

dginev wrote: post #160

Earlier quoted context omitted.

"Do we really need to manipulate a math expression?" If you don't, please don't say no one does. Having a math DOM allows for actual interactivity with mathematics, from highlighting, copying subtrees, embedding links, having on the fly computations/simplifications, etc. Not to mention add-on services like math indexing and search. Yes, we need to manipulate and machine-read math expressions, if we want to finally ta…

What I mean is "really need". In fact, there is also the possibility we want to highlight a portion of an image, copy a subimage, etc, but did our HTML tag designed like the way I mentioned? I am the author of a math search engine OPMES (tkhost.github.io/opmes), the search engine works pretty well without the knowledge of DOM structure of math expression. Actually MathML makes a lot inconvenient during OPMES developm…

So, if you can give me an ill-designed analogy, you think you're making a valid point?

Images are not mathematics, they have nothing to do with mathematics. If you take a look at SVG, you may be shocked to find you can do just as much decomposition of principle components as you can do with any DOM, just the way MathML allows you to.

Please substantiate the "works pretty well" claim about your search engine with some data. Your sense of inconvenience comes far from an objective argument.

How many people write HTML by hand? Generating it from a wide range of tools, richtext editors, markdown inputs, etc etc is much more common. Your analogies are just inaccurate.

Re: MathML is a failed web standard

#169

Post by dginev » Fri, Apr 08, 2016, 11:05 PM UTC

ga6840 wrote: post #165

dginev wrote: post #158

Earlier quoted context omitted.

I am the person behind generating the original NTCIR math datasets, and probably most of the research-produced MathML out there. We've recently presented that we have more than 350 million formulas from arXiv converted over to MathML, together with the rest of the papers as HTML5. As someone who has stared at arXiv TeX/LaTeX for years, I can testify you don't want to be looking at TeX math in actual latex documents,…

1. In NTCIR (main) dataset, I see many cases where does not contain an altext (and thus no TeX). I asked LaTeXML author Bruce Miller about this, he said LaTeXML will always put the same TeX string as an altext attribute on the . So I assume you guys are using some out-dated LaTeXML version? I really want to plead NTCIR to ensure the original LaTeX annotation is kept in main dataset, or please provide both MathML and…

1. Correct, the dataset was generated back in 2013 and will probably be regenerated for the next NTCIR issue.

2. There are annoying copyright issues with making the datasets available for public use. We're working with arXiv to resolve that, it's out of our control for now. It's a long-lasting frustration of mine that the datasets can't be simply made public.

3.You can omit anything you like from the MathML, there is no inferiority to omitting from TeX. "but maybe it is better than MWS" - prove it, submit to NTCIR, and beat everyone. Also, being better than MWS is not an argument that MWS should be denied the very data it needs to run. At the same time you can still obtain whatever degradation you need from the presentation MathML. Failing to recognize any claim to correctness than your own without any substantive proof is not a reasonable position and I urge you to reconsider.

"I have no idea why I need to read its lengthy spec, and I see no reason to write a MathML parser."

You don't need to write a parser, you can use an off-the-shelf parser for XML/HTML5 and handle the MathML reliably and appropriately. In fact you can reuse that from any open source search engine for math, MWS included. Writing a TeX parser on the other hand is something I will always roll my eyes at, since actual real world TeX is not something you can "parse", or do anything with reliably, unless you have a full TeX implementation underneath. Which is 1000x harder than using a parser to deal with MathML.

Finally, whining about NTCIR's UI being imperfect as a reason not to submit is just childish.

Re: MathML is a failed web standard

#170

Post by Mikhail_Edoshin » Sat, Apr 09, 2016, 7:14 AM UTC

TeMPOraL wrote: post #96

hansen wrote: post #74

Earlier quoted context omitted.

But that begs the question why web browsers don’t implement this. Is there a technical reason or did at some point some people decided that decent text rendering is irrelevant one should focus on other gimmicks?

The Web started as a tool for rendering text and occasional images pretty much as soon as it comes out of the wire. TeX on the other hand, was meant to be a typesetting package which compiles textual description of the document into a beautiful printable form. That compilation takes noticeable time even today, I imagine it was unacceptably slow for the web back before. Also in TeX you deal with paper sizes, in browse…

I remember an old publishing package, Ventura Publisher; the first time I saw it was 1990 or 91 maybe and then it was at v3. It was a very versatile publishing tool (all the perks of a professional typesetting package, tables, equations, floating illustrations; lots).

What's interesting is that it could separate the content and the styling information, so you could load the same content into two or more publications with different styles and get two different typesetting from the same source (e.g. one column small page for a book and two or three column large page for an article in a journal). And it was very easy to change the format or paper size: just change it and it would reflow the content accordingly. It didn't even took much time, as far as I can remember, and I'm talking about IBM PC 286 here.

So, truly, it's not a technical obstacle; web could have a much better typesetting engine working at high speed (and I'd say that it would be much better to imitate paged media instead of scrolling; scrolling is really inconvenient for reading). It's just that it grows wildly and in all directions at once; it tries to be a "semantic" storage, a rendering medium, and an application engine at the same time, and is not particularly good at any of this.