Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groundhog: Addressing the Threat That R Poses to Reproducible Research

datacolada.org

Post reply on HN
Print view View original

111–120 of 124 posts

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#111

Post by coliveira »

bonoboTP wrote: post #93
coliveira wrote: post #13

Earlier quoted context omitted.

That's something that eludes software people: reproducibility in science is the ability to create independent tests. Making software available, while useful, does very little for reproducibility from the scientific point of view.

It's a minimum standard, though. Of course the goal is reproducibility from a broader point of view, but that's not an excuse to do research in a one-off way where nobody is able to show how to get those numbers again, after a year or so from publication. The coding standards are often abysmally, unexpectedly terrible. Often not even the help of the original authors is enough to be able to produce the same figures fr…

The researcher's job is to properly do an experiment and document it in a paper. If we require more than this, then we will incur in damage to the scientific process for two reasons: (1) companies are not willing to make available software developed by their researchers, therefore they will publish even less; and (2) universities don't have money and staff to produce and maintain software at these standards, so professors will be required to publish less papers.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#112

Post by SoSoRoCoCo »

swsieber wrote: post #108
SoSoRoCoCo wrote: post #107

> The problem is that packages are constantly being updated, and sometimes those updates are not backwards compatible. Uh oh, someone just discovered the modern programming landscape! Python, Node, R, Rust, and other langs/OSes with package managers are at the mercy of volunteers who keep important packages healthy. Once issues stop being fixed, y'all better have local copies. This used to be predominantly an OS issu…

> The problem is that packages are constantly being updated, and sometimes those updates are not backwards compatible. > Python, Node, R, Rust, Correct me if I'm wrong, but for binary programs, a lock file easily mitigates these issue. I know Node and Rust both support lock files.

Yes, you're right. That's what makes lock files so important. I think we're past worrying about those wheels/npms/pkgs disappearing from the internet.

My concern is more about packages going stale and don't peer-match with other packages that evolve, or major versions that change results: not so much for R pkgs but there have been cases of major versions breaking existing projects, or requiring significant effort to update. (One example that zinged me is the FFI interface for Node. The "official" package hasn't been touched in years, and the "replacement", FFI-NAPI, is still has lots of open issues. We were using in-house fixes for some time.)

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#113

Post by huijzer »

epistasis wrote: post #101
roel_v wrote: post #59

Earlier quoted context omitted.

While this specific project does have a github page, the R world is 'complete amateur, avoid avoid avoid'. It's not really a 'programming language' in the way software engineers would see it. It's more a loose collection of stats functionality that is tied together with text interfactes in a way that somewhat looks like programming to the uninitiated. I mean, batch scripting is technically 'programming', and Excel (e…

> you can't do real software engineering This is completely, 100%, absolutely wrong. Of course you can. There's packages, with excellent software engineering structure, that are designed to include documentation and tests. R has so much good software engineering, that clever people with no software engineering background can easily make their own packages! And come on, the R language is a masterpiece. It's not cobble…

> Of course you can. There's packages, with excellent software engineering structure, that are designed to include documentation and tests.

For me, the problem with R is that the language is inconsistent. Many packages arose to address many problems, but they all feel like a hack on top of the core language. Take the whole Tidyverse; it just does dataframes from R core but then from the ground up. Now, users can choose between the core language dataframes and the Tidyverse dataframes. Same holds for plotting. The core issue, I think, is that the core language misses some essential features which other languages do have nowadays. For example, a type system. In R, since types are missing, everything is a table (dataframe) which I find just weird.

> It's not cobbled together like JavaScript or bash.

But also not as good as my favorite: Julia. Comparing it to Bash is like saying that its better than COBOL. We all know Bash is quite old, but for certain situations it just works.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#114

Post by cauthon »

resonantjacket5 wrote: post #18
cauthon wrote: post #15

This title is an exceedingly hot take for someone who wrote a new package manager. Also, it appears that Groundhog is itself a CRAN package and the author recommends installing with install.packages(). So is the author committing to never making any backwards incompatible updates to their new package?

Your take seems a bit 'hot' too? How else would you install the cran packages without using install.packages? Unless if you want them to recursively install it using groundhog but that seems unnecessary. As long as you have the timestamp it should work, though I assume there will be some edge case. What you're saying is like don't use pip because you don't install it using pip? Or don't use package-lock.json because…

No, I’m saying don’t call CRAN a “threat to reproducible science” and then make your solution a CRAN package

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#115

Post by epistasis »

huijzer wrote: post #113
epistasis wrote: post #101

Earlier quoted context omitted.

> you can't do real software engineering This is completely, 100%, absolutely wrong. Of course you can. There's packages, with excellent software engineering structure, that are designed to include documentation and tests. R has so much good software engineering, that clever people with no software engineering background can easily make their own packages! And come on, the R language is a masterpiece. It's not cobble…

> Of course you can. There's packages, with excellent software engineering structure, that are designed to include documentation and tests. For me, the problem with R is that the language is inconsistent. Many packages arose to address many problems, but they all feel like a hack on top of the core language. Take the whole Tidyverse; it just does dataframes from R core but then from the ground up. Now, users can choo…

The tidyverse is the benefit and the curse of metaprogramming, something that R takes from lisp, and something that has cursed (helped?) C++ since it was added.

As far as type systems, there's really two different types of "types": individual types objects that can have generic functions attached to them, etc. This is not as well known, and there are actually several object systems for typing:

http://adv-r.had.co.nz/OO-essentials.html

But these sort of objects are not quite as commonly created by programmers, because the second type of "types" are much more useful: data frames, which is kind of a vectorization of structs. This is what would be used in data oriented design, which is apparently much more common in modern game design.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#116

Post by Sebb767 »

IshKebab wrote: post #79
roel_v wrote: post #38

Apart from all the other considerations and problems with various types of package management, consider this: "Update January 6th, 2021 A reader alerted me to a bug with the current groundhog (version 1.1.0) where you cannot set the groundhog library to be a folder containing spaces in the name." So we are talking about software here that somehow made it to version 1.1 *without anyone ever using a directory with spac…

> So we are talking about software here that somehow made it to version 1.1 without anyone ever using a directory with spaces in it with it . This is extremely common, especially on Linux. Basically anything that uses things like Bash or CMake will almost certainly not work in directories containing spaces. Developers don't use paths containing spaces because it causes so many issues with badly written Bash scripts,…

> Developers don't use paths containing spaces because it causes so many issues with badly written Bash scripts, and as a result they don't test their code with paths containing spaces.

It is also a PITA to use when typing in a shell, as you need two characters ( \ + space ) instead of one. So even though my scripts can handle them, I still avoid them if possible.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#117

Post by [deleted] »

roel_v wrote: post #59
mattmanser wrote: post #48

Earlier quoted context omitted.

It doesn't even seem to be on GitHub, in fact the source doesn't seem to be listed anywhere on the project website. Which in our world would scream 'complete amateur, avoid, avoid, avoid', but perhaps it's different in the R world.

While this specific project does have a github page, the R world is 'complete amateur, avoid avoid avoid'. It's not really a 'programming language' in the way software engineers would see it. It's more a loose collection of stats functionality that is tied together with text interfactes in a way that somewhat looks like programming to the uninitiated. I mean, batch scripting is technically 'programming', and Excel (e…

[deleted]

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#118

Post by bonoboTP »

coliveira wrote: post #111
bonoboTP wrote: post #93

Earlier quoted context omitted.

It's a minimum standard, though. Of course the goal is reproducibility from a broader point of view, but that's not an excuse to do research in a one-off way where nobody is able to show how to get those numbers again, after a year or so from publication. The coding standards are often abysmally, unexpectedly terrible. Often not even the help of the original authors is enough to be able to produce the same figures fr…

The researcher's job is to properly do an experiment and document it in a paper. If we require more than this, then we will incur in damage to the scientific process for two reasons: (1) companies are not willing to make available software developed by their researchers, therefore they will publish even less; and (2) universities don't have money and staff to produce and maintain software at these standards, so profe…

"These standards" are pretty low. Currently it's a free-for-all chaos. Theoretically papers are reproducible from the documentation found in the paper but that is a lie. It is never reproducible just from the paper. Lots of stuff is done in the background that is not known to the reader. For all we know, they can even tweak their numbers to be 2% better and if someone can't get the results of the paper from the released code, the authors can just ignore it or say, the problem is not on their side, or that the paper numbers were generated with a slightly different code than the released version etc. I've seen this many times on Github, issues getting closed or deleted without comment etc. There is zero accountability.

It's slowly changing though but many people are grinding their teeth, because they can't torture the data as much if things are out in the open.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#119

Post by h2odragon »

mespe wrote: post #110

While I agree with many of the negative comments here about issues with how this is implemented, the tone of some comments is... not great. To the point that I would be reluctant to share work I do in R on Hacker News, which is not helping anyone. Just a reminder: https://news.ycombinator.com/newsguidelines.html

No kidding. Seems like an elegant solution to a potential problem to me. I've only used R at the "poke stick at numbers" level, but this would have been a useful addition to that trivial use.

Re: Groundhog: Addressing the Threat That R Poses to Reproducible Research

#120

Post by cat199 »

st1x7 wrote: post #53
cat199 wrote: post #51

Earlier quoted context omitted.

> there's software that people complain about and software that nobody is using. > The original quote is from Bjarne Stroustrup, the creator of C++ i find this ironic, given the 'popularity' (either way) of C++

I don't think it's ironic, the quote directly addresses the many criticisms towards C++.

ah whoops- completely misread it
Post reply on HN