Let’s make peer review scientific • Hacker News Board

Re: Let’s make peer review scientific

#31

Post by p4bl0 » Wed, Jul 06, 2016, 12:57 PM UTC

joelg wrote: post #25

p4bl0 wrote: post #24

Earlier quoted context omitted.

Hi! I really think what you are doing here is great and I hope you will succeed! Is there a way to test the platform without creating to much garbage? Like a sandbox version maybe?

Everything is open-sourced, and we've tried to document as much as possible to make it easy for people to host their own instances! Alternatively, you can just create arbitrarily many drafts without publishing any of them. https://github.com/pubpub/pubpub

If you don't mind I have a few questions :).

– PubPub doesn't seem to have math support, is that right? I think that should be a very high priority.

– LaTeX import is also really important (but this supposes that math is supported), is that on the roadmap?

– Related question: is there a way to work offline in my usual text editor and collaborate with my usual tools (git, svn, whatever) and then import my paper in PubPub?

Re: Let’s make peer review scientific

#32

Post by joelg » Wed, Jul 06, 2016, 1:02 PM UTC

p4bl0 wrote: post #31

joelg wrote: post #25

Earlier quoted context omitted.

Everything is open-sourced, and we've tried to document as much as possible to make it easy for people to host their own instances! Alternatively, you can just create arbitrarily many drafts without publishing any of them. https://github.com/pubpub/pubpub

If you don't mind I have a few questions :). – PubPub doesn't seem to have math support, is that right? I think that should be a very high priority. – LaTeX import is also really important (but this supposes that math is supported), is that on the roadmap? – Related question: is there a way to work offline in my usual text editor and collaborate with my usual tools (git, svn, whatever) and then import my paper in Pub…

Of course! Importing from markdown/latex/git, export to PDF/XML/etc, and lots more are coming (not in that order). We're still in very active development, so there's a lot left ahead on our roadmap.

If anyone is serious and passionate about helping out, we're hiring!

Re: Let’s make peer review scientific

#33

Post by p4bl0 » Wed, Jul 06, 2016, 1:05 PM UTC

joelg wrote: post #32

p4bl0 wrote: post #31

Earlier quoted context omitted.

If you don't mind I have a few questions :). – PubPub doesn't seem to have math support, is that right? I think that should be a very high priority. – LaTeX import is also really important (but this supposes that math is supported), is that on the roadmap? – Related question: is there a way to work offline in my usual text editor and collaborate with my usual tools (git, svn, whatever) and then import my paper in Pub…

Of course! Importing from markdown/latex/git, export to PDF/XML/etc, and lots more are coming (not in that order). We're still in very active development, so there's a lot left ahead on our roadmap. If anyone is serious and passionate about helping out, we're hiring!

Had I not landed a position in academia, this is really the type of project I would have love to work on if I had to do engineering rather than research ;).

But I really hope I can contribute as a user in the future :).

Re: Let’s make peer review scientific

#34

Post by joelg » Wed, Jul 06, 2016, 1:07 PM UTC

jessriedel wrote: post #26

joelg wrote: post #20

Shameless plug: I'm working at the MIT Media Lab on the PubPub project ( http://www.pubpub.org ), a free platform for totally open publishing designed to solve a lot of these problems: One is peer review, which, as some have already mentioned, needs to be done in an open, ongoing, and interactive forum. Making peer review transparent to both parties (and the public) makes everyone more honest. Another is the incentiv…

So I'm very interested in going beyond PDFs, but I can't for the life of me figure out what PubPub intends to do. I get that there's an online collaboration engine, as is offered by many places like Authorea, ShareLaTeX, Overleaf, and FindusWriter. What does PubPub do differently here?

Plugins and views!

We're currently working on a significant architecture refactor that will enable 3rd party plugins to view arbitrary data assets inline, like a reactive spreadsheet or a D3 visualization.

We're also working on deeper, richer hyperlinks between documents that would let you reference or quote other publications in the system with arbitrary resolution (down to the section/paragraph/sentence/number).

Re: Let’s make peer review scientific

#35

Post by nurettin » Wed, Jul 06, 2016, 1:09 PM UTC

randall wrote: post #14

ManlyBread wrote: post #13

Earlier quoted context omitted.

I disagree. Something similar exists on the wikipedia and it's still open to biases and agendas, it's just a matter of who is more persistent and which side the admins take.

But you don't need the 'gatekeeper'. Imagine if you had a 'trustability' factor that people could cryptographically sign as themselves. You could build a 'trust web', where papers link to other papers by an author. That author's trust ratings could go up based on the citer's evidence. There's clearly tech waiting to be built here, which might or might not be a startup.

The problem is not about proving your identity, it is about having the facilities, funds and connections for publishing papers which the gatekeepers seem to have. Cryptographic signing solves a different problem. And it is the least concerning one.

Re: Let’s make peer review scientific

#36

Post by isTravis » Wed, Jul 06, 2016, 1:11 PM UTC

p4bl0 wrote: post #31

joelg wrote: post #25

Earlier quoted context omitted.

Everything is open-sourced, and we've tried to document as much as possible to make it easy for people to host their own instances! Alternatively, you can just create arbitrarily many drafts without publishing any of them. https://github.com/pubpub/pubpub

If you don't mind I have a few questions :). – PubPub doesn't seem to have math support, is that right? I think that should be a very high priority. – LaTeX import is also really important (but this supposes that math is supported), is that on the roadmap? – Related question: is there a way to work offline in my usual text editor and collaborate with my usual tools (git, svn, whatever) and then import my paper in Pub…

Math support already there: http://i.imgur.com/tHKpNrB.png

We're keen on getting good offline support added, but (as you point out) there's still a lot to be done on the web side of things. We're pushing out some big updates that hopefully clarify the project and how to use it in a couple weeks.

Welcome to ideas on what good offline support would look like, or PRs if anyone wants to take a stab at it.

Our goal is for PubPub to be a public utility for scientific communication. It'll be non-profit and open source for as long as it lives (still a grad student project at the moment...) and free for anyone to publish whatever they like. I don't think we're nearly smart enough to know exactly what that looks like, so please do feel free to contribute ideas, code, or inspiration into what a public tool for science communication should be (this comments page is already wonderful in that regard).

Re: Let’s make peer review scientific

#37

Post by bixiou » Wed, Jul 06, 2016, 1:12 PM UTC

joelg wrote: post #20

Shameless plug: I'm working at the MIT Media Lab on the PubPub project ( http://www.pubpub.org ), a free platform for totally open publishing designed to solve a lot of these problems: One is peer review, which, as some have already mentioned, needs to be done in an open, ongoing, and interactive forum. Making peer review transparent to both parties (and the public) makes everyone more honest. Another is the incentiv…

If I might give my opinion on which features PubPub should implement next:

- adding "Math in-line $$" in Formatting menu

- enabling upload of LaTeX files

- enabling upload of PDFs, to store on PubPub existing research (combined by via.hypothes.is, one can annotate PDFs, which is in the spirit of PubPub)

- include some interesting features of SJS like grading, download as PDF, add to personal library...

I am impressed at the job you've done for a graduate project ! Best wishes :-)

Re: Let’s make peer review scientific

#38

Post by jessriedel » Wed, Jul 06, 2016, 1:13 PM UTC

joelg wrote: post #34

jessriedel wrote: post #26

Earlier quoted context omitted.

So I'm very interested in going beyond PDFs, but I can't for the life of me figure out what PubPub intends to do. I get that there's an online collaboration engine, as is offered by many places like Authorea, ShareLaTeX, Overleaf, and FindusWriter. What does PubPub do differently here?

Plugins and views! We're currently working on a significant architecture refactor that will enable 3rd party plugins to view arbitrary data assets inline, like a reactive spreadsheet or a D3 visualization. We're also working on deeper, richer hyperlinks between documents that would let you reference or quote other publications in the system with arbitrary resolution (down to the section/paragraph/sentence/number).

OK. Color me skeptical...

Re: Let’s make peer review scientific

#39

Post by p4bl0 » Wed, Jul 06, 2016, 1:19 PM UTC

isTravis wrote: post #36

p4bl0 wrote: post #31

Earlier quoted context omitted.

If you don't mind I have a few questions :). – PubPub doesn't seem to have math support, is that right? I think that should be a very high priority. – LaTeX import is also really important (but this supposes that math is supported), is that on the roadmap? – Related question: is there a way to work offline in my usual text editor and collaborate with my usual tools (git, svn, whatever) and then import my paper in Pub…

Math support already there: http://i.imgur.com/tHKpNrB.png We're keen on getting good offline support added, but (as you point out) there's still a lot to be done on the web side of things. We're pushing out some big updates that hopefully clarify the project and how to use it in a couple weeks. Welcome to ideas on what good offline support would look like, or PRs if anyone wants to take a stab at it. Our goal is for…

Ah, thanks. I tried math with single $ signs.

Re: Let’s make peer review scientific

#40

Post by joelg » Wed, Jul 06, 2016, 1:24 PM UTC

jessriedel wrote: post #38

joelg wrote: post #34

Earlier quoted context omitted.

Plugins and views! We're currently working on a significant architecture refactor that will enable 3rd party plugins to view arbitrary data assets inline, like a reactive spreadsheet or a D3 visualization. We're also working on deeper, richer hyperlinks between documents that would let you reference or quote other publications in the system with arbitrary resolution (down to the section/paragraph/sentence/number).

OK. Color me skeptical...

That’s fair. We think it’s important to be able to reference anything within the system with arbitrary resolution, so that’s our goal. Whether that implements well or not is certainly a technical challenge, but hard problems are the fun ones, right?