Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Challenge to scientists: does your ten-year-old code still run?

nature.com

Post reply on HN
Print view View original

81–90 of 487 posts

Re: Challenge to scientists: does your ten-year-old code still run?

#81

Post by qppo »

jhrmnn wrote: post #64
cube00 wrote: post #6

The day when code used to produce a paper must also be published can not come soon enough.

In all my papers the results were produced on multiple days (spanning months), with multiple versions of the code, and they are computationally too expensive to reproduce with the final version of the code. I'm trying to keep track of all the used versions, but given that there is no automated framework for this (is there?) and research involves lots of experiments, it's never perfect. Given this context, any ideas h…

That's no different than normal software engineering. We use version control software (VCS, like git) to deal with it. You can include your results in the tracked source.

For what it's worth, using results from outdated source code is extremely suspicious. This is a frequent problem in software development where we have tests or benchmarks based on stale code, and it's almost always incorrect. I would not trust your results if they are not created with the most up to date version of your software at all.

Re: Challenge to scientists: does your ten-year-old code still run?

#82

Post by BeetleB »

bumby wrote: post #60
dandelion_lover wrote: post #40

Earlier quoted context omitted.

As a theoretical physicist doing computer simulations, I am trying to publish all my code whenever possible. However all my coauthors are against that. They say things like "Someone will take this code and use it without citing us", "Someone will break the code, obtain wrong results and blame us", "Someone will demand support and we do not have time for that", "No one is giving away their tools which make their compe…

I’m curious, are dedicated software assurance teams a thing in your research area? Or is quality left up to the primary researchers?

> Or is quality left up to the primary researchers?

Individual researchers, and in many disciplines (like physics), there is almost no emphasis on quality.

I left academia a decade ago, but at the time all except one of my colleagues protested when version control was suggested to them. Some of these have code in the 30-40K lines.

Re: Challenge to scientists: does your ten-year-old code still run?

#83

Post by jnxx »

neutronicus wrote: post #51
throwanem wrote: post #12

Earlier quoted context omitted.

The fundamental problem here, as you note, is that scientists are rarely also engineers, and don't really share our desiderata. The point is to develop and publish a result, and engineering analysis code for resiliency is of secondary concern at best when that code isn't likely to need to be used again once the paper is finished. The "Software Carpentry" movement [1] has in the past decade tried to address this, as I…

Nah. The fundamental problem is that scientific code is produced by entry-level developers: 1. Paid below-market wages 2. With no way to move up in the organization 3. With lots of non-software responsibilities 4. With an expectation of leaving the organization in six years As long as the grunt work of science is done by overworked junior scientists whose careers get thrown to the wolves no matter what they do, you'r…

Even more fundamental is that there is no maintenance budget for important scientific libraries and tools. Somebody wrote them as part of their job, and the person who wrote it, is now working somewhere else.

Re: Challenge to scientists: does your ten-year-old code still run?

#84

Post by jnxx »

djsumdog wrote: post #16
dcolkitt wrote: post #9

I mean, Python 2->3 alone is gonna kill this challenge for most people.

You can always run old Python2 stuff in a Docker container, so long as the dependencies haven't disappeared.

As long as it does not use some CUDA hardware which is using tensorflow and Numba which is using a version of llvmlite which does not support Python2 any more.....

This isn't a theoretical example.

Re: Challenge to scientists: does your ten-year-old code still run?

#85

Post by dandelion_lover »

fsh wrote: post #48
paperwork wrote: post #41

Earlier quoted context omitted.

Can you describe a bit more about what is going on in the project? The file you linked is over 2.5k lines of c++ code, and that is just the “setup” file. As you say, this is supposed to be a statistical model, I expected this to be R, Python or one of the standard statistical packages. Why is there so much c++ code?

It's a Monte-Carlo simulation, not a statistical model. These are usually written in C++ for performance reasons.

Or Fortran.

Re: Challenge to scientists: does your ten-year-old code still run?

#86

Post by hpcjoe »

Short answer: Yes, my 30 year old Fortran code runs (with a few minor edits between f77 and modern fortran), as did my ancient Perl codes.

Watching the density functional theory based molecular dynamics zip along at ~2 seconds per time step on my 2 year old laptop, versus the roughly 6k seconds per time step on an old Sun machine back in 1991. I remember the same code getting down to 60 seconds per time step on my desktop R8k machine in the late 90s.

Whats been really awesome about that has been the fact that I've written some binary data files on big endian machines in the early 90s, and re-read them on the laptop (little endian) adding a single compiler switch.

Perl code that worked with big XML file input in the mid 2000s continues to work, though I've largely abandoned using XML for data interchange.

C code I wrote in the mid 90s compiled, albeit with errors that needed to be corrected. C++ code was less forgiving.

Over the past 4 months, I had to forward port a code from Boost 1.41 to Boost 1.65. Enough changes over 9 years (code was from 2011) that it presented a problem. So I had to follow the changes in the API and fix it.

I am quite thankful I've avoided the various fads in platforms and languages over the years. Keep inputs in simple textual format that can be trivially parsed.

Re: Challenge to scientists: does your ten-year-old code still run?

#87

Post by BeetleB »

Sebb767 wrote: post #24

As someone who worked with bits of scientific code: Does the code you write right now work on another machine might be the more appropriate challenge. If seen a lot of hardcoded paths, unmentioned dependencies and monkey-patched libraries downloaded from somewhere; just getting the new code to work is hard enough. And let's not even begin to talk about versioning or magic numbers. Similar to other comments I don't me…

> their job is not coding

To me, that's like a theoretical physicist saying "My job is not to do mathematics" when asked for a derivation of a formula he put in the paper.

Or an experimental physicist saying "My job is not mechanical engineering" when asked for details of their lab equipment (almost all of which is typically custom built for the experiment).

Re: Challenge to scientists: does your ten-year-old code still run?

#88

Post by jnxx »

nanddalal wrote: post #54

GitHub offers a free tier for GitHub actions with 2,000 Actions minutes/month [1]. This could be useful: 1. write some unit tests which don't use too much compute resources (so you can stick to the free tier) 2. package your code into a docker where the tests can be run 3. wire up the docker with tests to GitHub Actions This way now you have continuous testing and can make sure your codes keep running. References: [1…

> package your code into a docker

docker is not a general solution for this.

What is needed is a way to re-generate everything from source and from scratch.

Re: Challenge to scientists: does your ten-year-old code still run?

#89

Post by adornedCupcake »

"Python 2.7 puts “at our disposal an advanced programming language that is guaranteed not to evolve anymore”, Rougier writes1." Oh no. That's not at all what was intended. Regarding my own research: I'm doing theoretical biophysics. Often I do simulations. If conda stays stable enough, my code should be reproducible. There's however some external binaries(like lammps) I did not turn into a conda package yet. There's no official package that fits my use-case in conda since compilation is fine-grained to each user's needs.

Re: Challenge to scientists: does your ten-year-old code still run?

#90

Post by bumby »

BeetleB wrote: post #82
bumby wrote: post #60

Earlier quoted context omitted.

I’m curious, are dedicated software assurance teams a thing in your research area? Or is quality left up to the primary researchers?

> Or is quality left up to the primary researchers? Individual researchers, and in many disciplines (like physics), there is almost no emphasis on quality. I left academia a decade ago, but at the time all except one of my colleagues protested when version control was suggested to them. Some of these have code in the 30-40K lines.

I formerly worked in research, left and am now back in a quasi-research organization.

It’s bit disconcerting seeing how much quality is brushed aside particularly in software. Researchers seem to intuitively grasp how they need quality hardware to do their job, yet software rarely gets the same consideration. I’ve never been able to get many to come around to the idea that software should be treated the same as any other engineered product that enables their research

Post reply on HN