Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The arXiv of the future will not look like the arXiv

ar5iv.labs.arxiv.org

Post reply on HN
Print view View original

31–40 of 52 posts

Re: The arXiv of the future will not look like the arXiv

#31

Post by wolverine876 »

j-pb wrote: post #29
wolverine876 wrote: post #21

> the PDF is not a format fit for sharing, discussing, and reading on the web. PDFs are (mostly) static, 2-dimensional and non-actionable objects. It is not a stretch to say that a PDF is merely a digital photograph of a piece of paper. It is too far a stretch, murdering the poor subject: PDFs are the best format available for long-term information, such as research papers. They have the advantages of digital data: S…

None of those virtues hold in practice. I've worked both at public library digitization efforts and machine learning companies that did document ingestion and analytics. You always OCR the PDF visuals to get the text, because that's the only thing reliable about PDF. Everything else is often wrong, broken, or non-existent. By separating the meaning from the visual representation there is no incentive to keep the invi…

If I understand correctly, your comment addresses PDFs created from scanning paper. PDFs at arXiv are converted from LaTeX inputs, per the OP, and not via scanning and OCR; therefore they contain perfect renditions of the text.

>> Searchable, copy-able, transmittable, and data is extractable. They are also an open format, don't rely on a central service to be available, and they preserve presentation across platforms. They have metadata, and are annotatable and reviewable. And the PDF format is the best for long-term preservation, carefully designed to be readable in 50 years - partly because they preserve presentation across platforms - and that includes the metadata, annotations, and reviews.

> None of those virtues hold in practice.

> You always OCR the PDF visuals to get the text, because that's the only thing reliable about PDF. Everything else is often wrong, broken, or non-existent.

Which don't hold in practice? Are they not searchable? Is presentation not preserved? I use a lot of PDFs and they hold for me. PDFs are very popular, so they must work pretty well.

> SVG

Is there a standard way to do review and annotation, and is presentation preserved, for example when printing? Also, PDFs contain various image formats; do they contain SVG?

Re: The arXiv of the future will not look like the arXiv

#32

Post by pasadenasunset »

tempnow987 wrote: post #24

"sharing research via PDF must inevitably come to an end." Maybe instead of using the obsolete toolset arxiv provides, they could host their groundbreaking research on their own platform? The combination of ground breaking features and insightful commentary would draw users? Actually, many of the negatives they list are positives in my book. The latex barrier screens out a ton of garbage in my view - I'm on some soci…

I had a similar reaction. When you combine this concept with some of the fringe open science silliness, you are essentially telling scholars (what and how) they must share, AND (what and how) they are prohibited from sharing, leaving what is essentially a forced path:

"PDFs are prohibited, especially shared in private. All data, hypotheses, references, tables, code, must be presented in formats that are conducive to steali ^H^H^H^H replication and fostering a global science ethos of sharing."

"Authors who obey will have a beautiful platinum star printed next to their author nameplate. Extra star opportunities will exist for authors who announce their papers on Twitter with required levels of irony, hipness and verve!"

Re: The arXiv of the future will not look like the arXiv

#33

Post by pasadenasunset »

akvadrako wrote: post #2

It's fascinating to imagine what the arxiv of the future would look like. I imagine all scientific publications available on a distrusted block store, including raw emails, data and notes on a voluntary basis. Stuff that could be published would include reviews, corrections in version control fashion, and enough metadata to model scientific progress. What this article is describing sounds reasonable but not game chan…

Voluntary basis? You're an optimist, I see.

Re: The arXiv of the future will not look like the arXiv

#34

Post by pasadenasunset »

PeterisP wrote: post #23
kkfx wrote: post #19

A small proposal: why not a PopcornTime of papers? Witch means a distributed network (no matter if BitTorrent, ZeroNet, GNUNet, I2P or something else) to publish? That's the best freedom guarantee and just the mere number of nodes with a paper is a good metric about it's popularity, to avoid oblivion each uni/researcher can easily store and serve their own papers forever: files are small, so download is quick, not mu…

What problems does it solve for the authors? The features you describe above don't seem a problem in the current solutions; freedom and availability is a non-issue for authors, "to avoid oblivion each uni/researcher can easily store and serve their own papers forever" is a flaw not a feature (there are already far too many ways to do that, which only add extra burden to the authors if they want to "be everywhere" for…

This would and should be terrifying to any mid-career academic. The last thing needed is a complicated solution that solves no direct problem, YET offers plenty of "metrics" that one can attach all sorts of labels to, like "popularity".

Can you imagine some of the minds on academic Twitter holding a poll on article popularity? Leave science to the foul-tempered misanthropes, I say! j/k

Re: The arXiv of the future will not look like the arXiv

#35

Post by cmarschner »

ArXiv has this wonderful property: It works. It‘s simple. Everybody understands it. It is steady. People build tools around it (e.g. arxiv-sanity.com, Google scholar…) which make it even more useful. It has spin-offs like biorxiv that are catered to those communities. It is like a piece of infrastructure. Who cares aboud doi’s when the arxiv URL is already the standard? Yes it has its disadvantages, but none of them seems to justify to me to turn it upside down. Rather one could just add things, slowly, just like they‘re doing it.

Re: The arXiv of the future will not look like the arXiv

#36

Post by rhn_mk1 »

wolverine876 wrote: post #21

> the PDF is not a format fit for sharing, discussing, and reading on the web. PDFs are (mostly) static, 2-dimensional and non-actionable objects. It is not a stretch to say that a PDF is merely a digital photograph of a piece of paper. It is too far a stretch, murdering the poor subject: PDFs are the best format available for long-term information, such as research papers. They have the advantages of digital data: S…

> data is extractable

That's true for plain text (in the best case), but try extracting an equation, table or a diagram.

Stepping away from best case, PDFs in theory look the same everywhere, but turn into a mess on buggy implementations or differing rendering engines – due to the insistence on having a stable presentation, they assume positining and sizing always works, so when that fails, it fails worse than a buggy rendering of a presentation-agnostic document like an HTML page.

(In my experience, bugs either enter just before printing, or when displaying using JS-based renderers).

Re: The arXiv of the future will not look like the arXiv

#39

Post by j-pb »

wolverine876 wrote: post #31
j-pb wrote: post #29

Earlier quoted context omitted.

None of those virtues hold in practice. I've worked both at public library digitization efforts and machine learning companies that did document ingestion and analytics. You always OCR the PDF visuals to get the text, because that's the only thing reliable about PDF. Everything else is often wrong, broken, or non-existent. By separating the meaning from the visual representation there is no incentive to keep the invi…

If I understand correctly, your comment addresses PDFs created from scanning paper. PDFs at arXiv are converted from LaTeX inputs, per the OP, and not via scanning and OCR; therefore they contain perfect renditions of the text. >> Searchable, copy-able, transmittable, and data is extractable. They are also an open format, don't rely on a central service to be available, and they preserve presentation across platforms…

I'm not talking about scanned data. I'm talking about digitally born PDF, which have to be OCRed nevertheless because their text layer is unusable.

LaTex is one of the worst offenders when it comes to producing mangled text layers. Multi column text is often stored with both columns interleaving, or not at all. Verbatim is mangled, formulas are a hot mess. The text order between paragraphs is not preserved.

LaTex is an angstrom accurate type setting system and it's great for that, but it's abysimal at producing digital formats.

Could it produce better PDF documents? Sure. Does it do so, and do package authors care about any other layer except the printed visual one? No.

All those extra features you mention make your "still readable in 50 years" requirement go out the window pretty quickly. Long term archival is super tricky and considered an unsolved problem by libraries.

There's a reason ArXiv stores the LaTex as the canonical representation and not the PDF. The source code is simply a better archival format.

Re: The arXiv of the future will not look like the arXiv

#40

Post by kkfx »

pasadenasunset wrote: post #34
PeterisP wrote: post #23

Earlier quoted context omitted.

What problems does it solve for the authors? The features you describe above don't seem a problem in the current solutions; freedom and availability is a non-issue for authors, "to avoid oblivion each uni/researcher can easily store and serve their own papers forever" is a flaw not a feature (there are already far too many ways to do that, which only add extra burden to the authors if they want to "be everywhere" for…

This would and should be terrifying to any mid-career academic. The last thing needed is a complicated solution that solves no direct problem, YET offers plenty of "metrics" that one can attach all sorts of labels to, like "popularity". Can you imagine some of the minds on academic Twitter holding a poll on article popularity? Leave science to the foul-tempered misanthropes, I say! j/k

There is actually a direct problem, the same problem most people have and do not know until it bite that they have. For almost all is named "cloud", for Science is named reliance third party data, publications, that today might be accessible from someone else computer, tomorrow we do not know BUT we might still need them tomorrow, especially data and publications about big and discussed things, scandals involving big enterprises, governments etc

Actually too many trust that what's available today will be available in ten years, but such trust in IT terms is named a big weakness, our entire history if only digital on someone else computer is at risk of being easy changed in the future. For science that need is as bigger and in theory scientist should be smart and acculturated enough to comprehend.

I can imagine, for instance that in ten years a scandal about a today new drug emerge, and in ten years original trial evidence is gone, on purpose and the scandal became then polemics just because there's nothing to really analyze so no one can really tell what happen.

I can imagine, in shorter time, that a day a scientist from a certain country found it's "own" paper collection along all it's notes, stored on some cloud reference manager gone, perhaps because his/her country have political disagreement with the cloud provider country or because it's university can't pay anymore access to such private and high-price platform.

If just we REALLY want peer review we can't even trust public places like HAL, we need to have data in our own hands, since they quickly became too much the sole alternative is having them on a distributed network where anyone hold some data and it's hard to believe a cartel to erase/temper them with a sufficiently large number of known nodes (like scientist authenticating themselves with GNUPG/PGP signatures with the relevant PKI infra and cross-signed keys behind.

Post reply on HN