Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groundhog: Addressing the Threat That R Poses to Reproducible Research

datacolada.org

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#71

Post by vharuck »

roel_v wrote: post #59
mattmanser wrote: post #48

Earlier quoted context omitted.

It doesn't even seem to be on GitHub, in fact the source doesn't seem to be listed anywhere on the project website. Which in our world would scream 'complete amateur, avoid, avoid, avoid', but perhaps it's different in the R world.

While this specific project does have a github page, the R world is 'complete amateur, avoid avoid avoid'. It's not really a 'programming language' in the way software engineers would see it. It's more a loose collection of stats functionality that is tied together with text interfactes in a way that somewhat looks like programming to the uninitiated. I mean, batch scripting is technically 'programming', and Excel (e…

This argument seems elitist. R is more than just technically Turing complete.

It's definitely a specialized language. It's not the go-to for managing servers or anything with a lot of I/O, but it has those capabilities because they're useful for managing projects. And I'd be hard-pressed to justify using a language for statistical analysis if it doesn't focus on statistical analysis. It'd be like rolling my own cryptography.

You need to differentiate between "base R" (everything that comes with a new install) and community-contributed packages. Base R is amazingly reliable. It has detailed documentation[0].

User-package land is more of a Wild West, that's true. I would personally not use anything that's not on CRAN unless I can walk up to the maintainer's desk (in non-pandemic times).

[0] https://cran.r-project.org/manuals.html

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#72

Post by wodenokoto »

Wow, that's a lot of pessimism for a fairly elegant solution to the fact that almost no R code has package versioning defined.

I think the major sales point here is:

> A nice feature of groundhog is that it makes 'retrofitting' existing code quite easy. If you come across a script that no longer works, you can change its library() statements for groundhog.library() ones, using as the groundhog.day the date the code was probably written (say when it was posted on the internet), and it may work again.

I don't know how good ratpack is now a days. I've never met an R application that uses it, but at my old work, we would take a dated snapshot of CRAN at the beginning of every new project. If we needed to update a package we could then "update CRAN" for that project. When productionising a project it would be frozen to a date in CRAN.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#73

Post by nojito »

wodenokoto wrote: post #72

Wow, that's a lot of pessimism for a fairly elegant solution to the fact that almost no R code has package versioning defined. I think the major sales point here is: > A nice feature of groundhog is that it makes 'retrofitting' existing code quite easy. If you come across a script that no longer works, you can change its library() statements for groundhog.library() ones, using as the groundhog.day the date the code w…

>Wow, that's a lot of pessimism for a fairly elegant solution to the fact that almost no R code has package versioning defined.

This isn't true.

https://mran.microsoft.com/documents/rro/reproducibility

https://rstudio.github.io/packrat/

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#74

Post by roel_v »

vharuck wrote: post #71
roel_v wrote: post #59

Earlier quoted context omitted.

While this specific project does have a github page, the R world is 'complete amateur, avoid avoid avoid'. It's not really a 'programming language' in the way software engineers would see it. It's more a loose collection of stats functionality that is tied together with text interfactes in a way that somewhat looks like programming to the uninitiated. I mean, batch scripting is technically 'programming', and Excel (e…

This argument seems elitist. R is more than just technically Turing complete. It's definitely a specialized language. It's not the go-to for managing servers or anything with a lot of I/O, but it has those capabilities because they're useful for managing projects. And I'd be hard-pressed to justify using a language for statistical analysis if it doesn't focus on statistical analysis. It'd be like rolling my own crypt…

shrug. It's largely opinion-based, I guess. My pet peeve (which also illustrates my point, but again, in an opinion-based way): there is no documented, 'officially supported' way to get the path of the current script in R. That is not a problem for amateur programmers who don't think about things like robustness, distribution etc, and it's needlessly complicated and bolted on in SAS, too. But it's still silly and indicative of R's typical use cases. Excel is reliable and well documented too, and I still wouldn't call even complicated workbooks 'software engineering'.

And CRAN... well... let's just say that people used to point to CPAN as a strength of Perl, too... All that sort of archives, after the first few years which comprise mostly of contributors with deep knowledge and who can produce high quality libraries, turn into dumping grounds for trivial half-assed 'libraries' under the guise of 'community contributions'. Example: try to do trivial compound interest simulations in R. So basic that it's barealy worth calling 'finance'. There are (at least) three packages on CRAN that claim to do this, except that (depending on which variable in the equation you want to solve for) they all provide only part of the solution, in mostly incompatible ways. And this is because very few of the people putting code into CRAN know how to... well... write good code. This is not an indictment of those people; many of them are much more intelligent than a bunch of us combined. It's just that for them coding is a byproduct, and with good intentions they share what has been useful for them, it just leads to a situation of 'in the land of the blind one eye is king'.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#75

Post by vharuck »

scottmcdot wrote: post #67
resonantjacket5 wrote: post #18

Earlier quoted context omitted.

Your take seems a bit 'hot' too? How else would you install the cran packages without using install.packages? Unless if you want them to recursively install it using groundhog but that seems unnecessary. As long as you have the timestamp it should work, though I assume there will be some edge case. What you're saying is like don't use pip because you don't install it using pip? Or don't use package-lock.json because…

Someone correct me if I'm wrong, but can't you copy and paste the package folder into your libpath directory and R can load it that way with actually running install.packages()?

Usually, yes. However, it is possible for a package to have code that only runs when it is installed. If you just copy-paste, it won't be run.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#76

Post by vharuck »

notafraudster wrote: post #17
anderscarling wrote: post #6

I’ve yet to use it personally, but renv [1] seems to try to solve the reproducible builds problem in a way more similar to other modern package managers (e.g. by generating a lockfile). This approach enables stricter validations against tampering with the package repositories as a hash of the package can be stored in the lockfile, however it is obviously a bit more complex to use than the groundhog approach. [1]: htt…

Agreed that renv is a better solution here. Even the example code for Groundhog is not written in idiomatic R which does not inspire confidence. Simonssohn is a legend in transparent research but not primarily a coder or software tool contributor (take a look at the source for p-curve if you want to see what I mean) and I think a secondary threat to reproducibility is relying on tools that end up abandoned or depreca…

>Even the example code for Groundhog is not written in idiomatic R which does not inspire confidence.

Not to mention the given example for irreproducibility in base R looks at code that would be a bug in the script for 3.6. It's only useful to keep this reproducible if I'm debugging the script.

And, in this case, anyone who's proficient with R would recognize this problem from personal experience or the many warnings in tutorials. I usually wouldn't shoot down a given example as though it disproved the existence of any example, but I don't know if there is another example. Unless old code relied on undocumented or contrary-to-documented behavior.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#78

Post by ekianjo »

st1x7 wrote: post #33

Either I'm misunderstanding or this is a non-problem. You can specify older versions of a package when you install it. You can also manage them with packrat. As long as researchers share their language and package versions, you can fully reproduce their environment. (And the base language is really stable, almost to a fault.) This is just a bad way for the author to promote their own library for dealing with this. Th…

Packrat is deprecated now. It is recommended to use renv instead.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#79

Post by IshKebab »

roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

> So we are talking about software here that somehow made it to version 1.1 without anyone ever using a directory with spaces in it with it.

This is extremely common, especially on Linux. Basically anything that uses things like Bash or CMake will almost certainly not work in directories containing spaces.

Developers don't use paths containing spaces because it causes so many issues with badly written Bash scripts, and as a result they don't test their code with paths containing spaces.

Bash and CMake and similar hacked together languages have very error-prone quoting rules that make it very easy to accidentally make something work with paths without spaces but fail on paths with spaces.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#80

Post by _Wintermute »

st1x7 wrote: post #33

Either I'm misunderstanding or this is a non-problem. You can specify older versions of a package when you install it. You can also manage them with packrat. As long as researchers share their language and package versions, you can fully reproduce their environment. (And the base language is really stable, almost to a fault.) This is just a bad way for the author to promote their own library for dealing with this. Th…

Correct me if I'm wrong but specifying an older package version in R still pulls the newest packages from CRAN for any dependencies, which is quick way to run into a load of incompatibilities.

I've not tried renv yet but packrat was a pretty poor solution.

Post reply on HN