“Hey guys, we need to train a large language model to understand all of science. All we have is a stack of old papers from the dawn of time and we need to convert them into LaTeX…”
Nougat: Neural Optical Understanding for Academic Documents
21–30 of 30 posts
Re: Nougat: Neural Optical Understanding for Academic Documents
#22Earlier quoted context omitted.
HTML and various javascript libraries like mathjax or other libraries for charts and graphs. > Doesn't seem like HTML would fly, either, given all the kinds of things that get embedded into PDF. That's ironic. Browser PDF readers, at least open source ones, render PDFs as HTML using javascript. At least I'm sure about FF because I just checked that text from a native-digital pdf showed up in the DOM in developer tool…
> HTML and various javascript libraries like mathjax or other libraries for charts and graphs. How do you email that to someone as an attachment? Can you embed all of that stuff into a single .html file?
> Can you embed all of that stuff into a single .html file?
Technically yes, but there are two problems.
First is that inline style and scripts are a potential security vulnerability.
Second is that if someone does not inline everything and instead references css/js from the web the document will stop rendering correctly when those resources go offline.
Re: Nougat: Neural Optical Understanding for Academic Documents
#23Semi offtopic, but don't pretend you're making an acronym when the letters aren't the first ones in each word.
Or, as I like to call it, SOUNDYMAREHEATRONER.
Re: Nougat: Neural Optical Understanding for Academic Documents
#24Funnily enough the Example Page 1 is wrong. Rendering du^n as du^*, and then nu^n-1 as nw^*-1. It is impressive but...it really feels like those are the details that really really matter.
The main issue with OCR of anything with math in it is always that it has to be not 99.99% but 100% correct. Which is probably not possible.
Re: Nougat: Neural Optical Understanding for Academic Documents
#25Earlier quoted context omitted.
HTML and various javascript libraries like mathjax or other libraries for charts and graphs. > Doesn't seem like HTML would fly, either, given all the kinds of things that get embedded into PDF. That's ironic. Browser PDF readers, at least open source ones, render PDFs as HTML using javascript. At least I'm sure about FF because I just checked that text from a native-digital pdf showed up in the DOM in developer tool…
> HTML and various javascript libraries like mathjax or other libraries for charts and graphs. How do you email that to someone as an attachment? Can you embed all of that stuff into a single .html file?
Sure, epub.
You could (or maybe you can't, but ebook readers should allow you to) disable any network access without explicit confirmation, so the javascript can't do anything evil other than modify the ebook being displayed. If you can't do that, that's up to ebook readers to solve, and not a flaw with epubs.
Re: Nougat: Neural Optical Understanding for Academic Documents
#26Earlier quoted context omitted.
There's nothing immutable about pdfs. If you have an "original" document, it'll always hash to whatever it hashes to. I fail to see the point. You can cite md5 hashes on LG the same whether they're pdfs or epubs or, heaven forbid, azw3 (amazon's proprietary epub-like format). What's the obsession with "looking the same everywhere"? Page references: this shouldn't be a thing. Academia has already solved this problem f…
that's all missing the point. mhtml would somewhat fit part of the bill of what PDF offers: a single downloadable "file" you can archive or forward and you know: the recipient will see exactly what you saw. however the mhtml doesn't look the same, depending on the device. and looking.exactly the same helps a great deal in convincing a judge that we all talk about the same.thing. get me right. I hate PDF with all pass…
I agree PDF is not entirely useless. I don't mind it that much for papers, but I don't see how it's any better than responsive formats. It's obviously important for presentations where exact visual relationships between items is fixed, or flyers (things that inherently get published on dead trees—leaving layout to an html rendering engine in that case is not great), or other things like that.
I still don't understand why it needs to look exactly the same. I get that habitually people say "turn to page X and look 2/3 down the page for the line starting "The quick brown fox jumped", but with digital documents that's not in my experience how anything works. You just say "the sentence starting with 'The quick brown fox'", and everyone can search for it in a few seconds.
If an official proceeding needs to be sure everyone's working from the same document, they can distribute, or publish hashes of, an epub or mhtml the same as they can for a pdf. There's no assurance that two pdfs that you think are the same document are actually the same document, any more than two epubs would be.
Re: Nougat: Neural Optical Understanding for Academic Documents
#27I made a quick test:
1. GPU resoursce consuming. 16G~
2. I test some document other than English, it is poor.
Re: Nougat: Neural Optical Understanding for Academic Documents
#28This is great, but when is academia, business and government going to finally get off PDF as a typical standard? It's awful, not adaptive for mobile, and a pain in the ass to work with for any kind of development.
not adaptive for mobile I look at pdfs on my phone all the time, it's great. 'Optimized for mobile' usually means oversized fonts and a shitty UI so I get RSI in my thumb from endless scrolling. PDF is kind of an ugly format, but the problem with realtime text flow etc. is that designers are (at the behest of clients) are always trying to look visually distinct and as a result nothing is standardized or predictable a…
It works, but that doesn't mean it's good. A document designed to be printed on an A4 page isn't easy to read when scaled down to the size of a phone, especially for people with suboptimal eyesight, and if you zoom in to read the text, you need to scroll horizontally for every line you want to read
Re: Nougat: Neural Optical Understanding for Academic Documents
#29it's interesting to see they didn't pick their examples to exclude all failures. I find that really great and it should be done more often
Re: Nougat: Neural Optical Understanding for Academic Documents
#30Earlier quoted context omitted.
not adaptive for mobile I look at pdfs on my phone all the time, it's great. 'Optimized for mobile' usually means oversized fonts and a shitty UI so I get RSI in my thumb from endless scrolling. PDF is kind of an ugly format, but the problem with realtime text flow etc. is that designers are (at the behest of clients) are always trying to look visually distinct and as a result nothing is standardized or predictable a…
It works, but that doesn't mean it's good. A document designed to be printed on an A4 page isn't easy to read when scaled down to the size of a phone, especially for people with suboptimal eyesight, and if you zoom in to read the text, you need to scroll horizontally for every line you want to read
I'm aware of all that, although your complaint about zooming in is easily fixable by turning the phone sideways. My point is that PDF remains popular because it's predictable and because print layout standards are generally better than dynamic ones. That's the problem to address: people's desire to read longform text in a consistent way without being continually derailed by interactive cruft.