Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groundhog: Addressing the Threat That R Poses to Reproducible Research

datacolada.org

Post reply on HN
Print view View original

61–70 of 124 posts

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#61

Post by _Wintermute »

GlennS wrote: post #3

I know of two other existing solutions to this, although I don't know enough to compare. I don't think either of these tick all the author's boxes. Microsoft MRAN https://mran.microsoft.com/ > For the purpose of reproducibility, MRAN hosts daily snapshots of the CRAN R packages and R releases as far back as Sept. 17, 2014. MRAN doesn't seem to be very well known or used in the R community, but I don't really know why…

MRAN has saved my bacon more than once when I need to replicate some R environment written years ago. The package management in R really is terrible.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#62

Post by jcelerier »

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

> This can be interpreted in two ways: either very few people have spaces in their paths

it's been years since I've seen anyone doing that - a main reason, is that a very widely used dev tool, make, does not handle spaces in paths:

http://savannah.gnu.org/bugs/?712

thus leading to inertia in the whole ecosystem - if make does not support spaces in paths, why bother

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#63

Post by roel_v »

tpxl wrote: post #46
roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

Could also be that the package manager doesn't use spaces and most people use package managers? Ie maven will create a folder structure like "/home/user/.m2/repository/com/example/example.jar" which will never have spaces unless the username has spaces (Can linux usernames have spaces?).

On Unixy systems, spaces are uncommon because so little software can deal with them, so that people are trained from the very beginning to treat spaces like the plague. I do it too - I've been burned by treatment of spaces in shitty 0.x level software so many times (25+ years ago) that I now have an intuitive aversion of anything with spaces.

Spaces in filenames are a reality though, especially on Windows (where the home directory itself used to have spaces in it, and also where many home directories on corporate networks are on network drives and start with \\), and any software that can't deal with those kinds of paths has just not been exposed to much (if any) real world use. That was the point I was trying to make - software that can't handle anything but the most bog-standard path names in its core configuration is 'hey guys look at what I hacked up yesterday evening' quality at best. (yes yes it is possible to imagine exceptions, like software that is decades old and ported across platforms; I'm talking about something new that is meant to solve a general problem).

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#64

Post by CJefferson »

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

If you start discarding software which has problems with a space in a directory name, you should start with libtool, at which point you can't build significant chunks of the Linux ecosystem.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=193163

I hit this when trying to test libgmp (as an example of an important library you would lose).

This means in practice you can't really build most software which uses configure scripts and libraries in a directory with a space -- this may well be what they are hitting.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#65

Post by SCLeo »

st1x7 wrote: post #36
bsza wrote: post #21

Earlier quoted context omitted.

I think it's more like a Wayback Machine for R programs, since the author of a science paper isn't required to use groundhog. You can just provide it the date the article was published, which you already know, and it reconstructs how the program worked on that day. Also, because groundhog isn't made for the author to use, whether or not the interface changes is irrelevant. You'll never encounter library(groundhog) in…

> and it reconstructs how the program worked on that day. It reconstructs how the fully updated version of everything worked that day which isn't necessarily the same as the researcher's environment. It's a horrible idea to use dates instead of package versions for this. The author's library doesn't solve the problem it claims to solve.

If I am understanding this correctly, the problem is that the paper authors do not provide a specific version or a package.json equivalent. In that case, using dates seem to be the only choice.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#66

Post by st1x7 »

SCLeo wrote: post #65
st1x7 wrote: post #36

Earlier quoted context omitted.

> and it reconstructs how the program worked on that day. It reconstructs how the fully updated version of everything worked that day which isn't necessarily the same as the researcher's environment. It's a horrible idea to use dates instead of package versions for this. The author's library doesn't solve the problem it claims to solve.

If I am understanding this correctly, the problem is that the paper authors do not provide a specific version or a package.json equivalent. In that case, using dates seem to be the only choice.

Even if that's the case, using dates isn't a solution because dates don't give you the build that the researcher used. Date of publication is different from the date when the code ran and there is no guarantee that the researcher ran the latest version of every dependency that was available to them anyway. In fact that's very unlikely considering that some their libraries might require older versions. It might not even be possible to take the latest version of every package and use them in the same environment.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#67

Post by scottmcdot »

resonantjacket5 wrote: post #18
cauthon wrote: post #15

This title is an exceedingly hot take for someone who wrote a new package manager. Also, it appears that Groundhog is itself a CRAN package and the author recommends installing with install.packages(). So is the author committing to never making any backwards incompatible updates to their new package?

Your take seems a bit 'hot' too? How else would you install the cran packages without using install.packages? Unless if you want them to recursively install it using groundhog but that seems unnecessary. As long as you have the timestamp it should work, though I assume there will be some edge case. What you're saying is like don't use pip because you don't install it using pip? Or don't use package-lock.json because…

Someone correct me if I'm wrong, but can't you copy and paste the package folder into your libpath directory and R can load it that way with actually running install.packages()?

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#68

Post by bayindirh »

roel_v wrote: post #42
bayindirh wrote: post #40

Earlier quoted context omitted.

As I can see from the researchers in our cluster and my own academic research, most people still avoid spaces in paths and files like the plague. YMMV of course.

Of course, and so do I. But nobody ever even encountering the situation and/or bothering to report it, that's a whole different matter.

My guess is people are encountering the situation, working around it and calling a day. Maybe a little note here and there but, I don't think someone would report it due to a couple of reasons.

First of all, I don't think people report this type of stuff because they don't know how to report it, and secondly think it doesn't need to support this use case anyway since space is a latecomer to naming and path game.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#69

Post by jbullock35 »

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

A further concern: the repository for this R package [1] doesn't include any test files. Am I right to think that we should be wary of R packages that don't have any unit tests?

https://github.com/CredibilityLab/groundhog

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#70

Post by fjcp »

bayindirh wrote: post #40
roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

As I can see from the researchers in our cluster and my own academic research, most people still avoid spaces in paths and files like the plague. YMMV of course.

As a Linux user I can relate to that. I always avoid spaces in folders and filenames as they make it more annoying to manipulate them using command line tools. Years later I carried this habit to whatever OS I am using.
Post reply on HN