Groundhog: Addressing the Threat That R Poses to Reproducible Research • Hacker News Board

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#41

Post by kristaps » Thu, Jan 07, 2021, 9:15 AM UTC

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

Don't remember the source and probably misquoting, but I like this truism: there's software that people complain about and software that nobody is using.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#42

Post by roel_v » Thu, Jan 07, 2021, 9:17 AM UTC

bayindirh wrote: post #40

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

As I can see from the researchers in our cluster and my own academic research, most people still avoid spaces in paths and files like the plague. YMMV of course.

Of course, and so do I. But nobody ever even encountering the situation and/or bothering to report it, that's a whole different matter.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#43

Post by st1x7 » Thu, Jan 07, 2021, 9:23 AM UTC

kristaps wrote: post #41

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

Don't remember the source and probably misquoting, but I like this truism: there's software that people complain about and software that nobody is using.

The original quote is from Bjarne Stroustrup, the creator of C++. The quote also doesn't apply here. (You can't just use it to excuse any problem with software that you come across). The author of the article and the library in it just seems out of their depth in many ways.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#44

Post by dstick » Thu, Jan 07, 2021, 9:38 AM UTC

bayindirh wrote: post #40

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

As I can see from the researchers in our cluster and my own academic research, most people still avoid spaces in paths and files like the plague. YMMV of course.

If my own hobby python projects are anything to go by, there aren’t even folders ;-)

I have a friend who taught herself R for her research and it was basically one big procedural codebase.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#45

Post by GlennS » Thu, Jan 07, 2021, 9:42 AM UTC

chalst wrote: post #20

GlennS wrote: post #3

I know of two other existing solutions to this, although I don't know enough to compare. I don't think either of these tick all the author's boxes. Microsoft MRAN https://mran.microsoft.com/ > For the purpose of reproducibility, MRAN hosts daily snapshots of the CRAN R packages and R releases as far back as Sept. 17, 2014. MRAN doesn't seem to be very well known or used in the R community, but I don't really know why…

The article discusses MRAN in footnote 5, when arguing against the MRAN-based 'checkpoint' approach. Nixpkg/Nixos is obviously a useful technology for reproducibility, but note that the output of Nix scripts can depend on the time the system was built, the contents of URLs and the system architecture unless care is taken.

So it does, I missed that!

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#46

Post by tpxl » Thu, Jan 07, 2021, 10:02 AM UTC

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

Could also be that the package manager doesn't use spaces and most people use package managers?

Ie maven will create a folder structure like "/home/user/.m2/repository/com/example/example.jar" which will never have spaces unless the username has spaces (Can linux usernames have spaces?).

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#47

Post by samch93 » Thu, Jan 07, 2021, 10:02 AM UTC

Can recommend the paper "A Reproducible Data Analysis Workflow with R Markdown, Git, Make, and Docker" by Peikert and Brandmaier [1], which shows a much more robust approach to reproducibility.

[1] https://psyarxiv.com/8xzqy/

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#48

Post by mattmanser » Thu, Jan 07, 2021, 10:14 AM UTC

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

It doesn't even seem to be on GitHub, in fact the source doesn't seem to be listed anywhere on the project website.

Which in our world would scream 'complete amateur, avoid, avoid, avoid', but perhaps it's different in the R world.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#49

Post by kwertzzz » Thu, Jan 07, 2021, 10:17 AM UTC

st1x7 wrote: post #33

Either I'm misunderstanding or this is a non-problem. You can specify older versions of a package when you install it. You can also manage them with packrat. As long as researchers share their language and package versions, you can fully reproduce their environment. (And the base language is really stable, almost to a fault.) This is just a bad way for the author to promote their own library for dealing with this. Th…

I full agree. With version number you can have a good sense whether a package update breaks your code or not (as long the package authors following semantic versioning).

I think in julia this problem is solved quite nicely with the Project.toml (list of package that you directly dependent) and Manifest.toml file (the version numbers of the complete dependency tree which is automatically generated).

It seems that in groundhog you declare only direct dependencies. Is there a way to store the full dependency tree in R ?

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#50

Post by qwantim1 » Thu, Jan 07, 2021, 10:19 AM UTC

mattmanser wrote: post #48

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

It doesn't even seem to be on GitHub, in fact the source doesn't seem to be listed anywhere on the project website. Which in our world would scream 'complete amateur, avoid, avoid, avoid', but perhaps it's different in the R world.

No, I think you’re correct. Incomplete source is bad in any world.

Unfortunately, it’s that world we live in for pretty much everything.

Reproducibility? What if all of the source were to depend on part of a CPU instruction set that we stop using? How long must things be reproducible? We don’t even make lab equipment exactly like we used to with the experiments our current sciences are based on.

However, I give a thumbs up to Groundhog for trying to do the right thing.