Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evaluation of C, Go, and Rust in the HPC environment [pdf]

octarineparrot.com

Post reply on HN
Print view View original

31–40 of 65 posts

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#31

Post by ohquu »

chrisseaton wrote: post #29
physguy1123 wrote: post #24

Earlier quoted context omitted.

There is still a lot of Fortran being written, but a lot of it is on legacy projects (scientific codes have a habit of lasting a long time). While there are some cool things like coarray Fortran and some groups require that everything be done in Fortran, C++ is a favorite for new projects afaik.

Why do scientists call programs 'codes'? It's something that seems to be particularly common in HPC, and quite rare elsewhere.

It's easier and more precise to say something like "run the quantum Monte Carlo code" than "run the quantum Monte Carlo model" or "run the quantum Monte Carlo software."

A quantum Monte Carlo code will of course include a model, but I think people don't want to call it "software" because it's so research-grade and janky. "Program" seems better, but I think that implies that it's a static thing (not in a constant state of development).

The plural "codes" is used because usually a research team has historically implemented a bunch of models into disparate codebases.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#32

Post by Manishearth »

gaius wrote: post #14

Any HPC language evaluation that doesn't include vs FORTRAN is pointless. It's still king of computation for a reason.

... the reason is mostly legacy. All your libraries are written in fortran. Everyone learned fortran in the past and they don't want to switch.

As an example, I've seen physicists use fortran for parsing and transforming the output of a program. The cluster at our department has an awesomely updated ifort but no python2.7 (I think it has 2.2 or 2.3 or something). C++ isn't used directly either.

But yeah, there should have been fortran and some other languages there.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#33

Post by hurin »

Dewie3 wrote: post #30
bsdetector wrote: post #10

The author has a "lack of equal experience in the three evaluated languages", especially C, but is comparing them for development time and SLOC based on a sample size of one developer and one very small program. He rewrites a distributed program as a threaded one, because Rust and Go didn't have a distributed computing library, but says Go and Rust have "similar performance" in HPC. He says he was surprised at the C…

It's a bachelor thesis... probably doesn't aspire to be the new authority on HPC. :)

Yes, because the point is being able to write (on a resume): successfully presented thesis on "Evaluation of C, Go, and Rust in the HPC environment".

The advisors should have encouraged a smaller and more manageable scope with a more thorough methodology where the results would actually be worth something.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#34

Post by runlevel1 »

gaius wrote: post #14

Any HPC language evaluation that doesn't include vs FORTRAN is pointless. It's still king of computation for a reason.

Experiments aren't usually written in FORTRAN.

The programmer is by far the slowest part of HPC. It often makes sense to use the language you're fastest at instead of the language that will run the fastest.

It's very common for experiments to be written in languages not know for their performance (Java, Matlab, Perl, and Python for instance).

For reference, here are some of the packages commonly used in experiments[1]:

    BLAS        Fortran                 http://www.netlib.org/blas/#_software
    NCBI BLAST  C++                     ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.2.30+-src.zip
    BFAST       C                       http://sourceforge.net/projects/bfast/
    BioPerl     Perl                    https://github.com/bioperl/bioperl-live
    Bowtie      C++, C                  http://sourceforge.net/projects/bowtie-bio/files/bowtie/
    Clustal     C, C++                  http://www.clustal.org/omega/#Download
    cp2k        Fortran                 https://github.com/cp2k/cp2k
    Gromacs     C                       https://github.com/gromacs/gromacs
    HTSeq       Python                  https://pypi.python.org/pypi/HTSeq
    MUSCLE      C++                     http://www.drive5.com/muscle/downloads.htm
    MrBayes     C                       http://sourceforge.net/p/mrbayes/code/HEAD/tree/
    OpenFOAM    C++                     https://github.com/OpenFOAM/OpenFOAM-2.3.x
    SAMtools    C                       https://github.com/samtools/samtools
    SNAP        C                       http://korflab.ucdavis.edu/software.html
    fftw3       C                       https://github.com/FFTW/fftw3
[1]: I went down this list and picked some of the ones I remember using (been out of the HPC world for a couple years): https://portal.tacc.utexas.edu/software It'll be a bit skewed towards genomics.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#35

Post by deadgrey19 »

Keep in mind, this is a bachelors thesis. It's not a peer reviewed research article in a respected systems journal or conference. I mean this in both a positive and a negative sense. Positive in sense that the purpose of a bachelors thesis is to introduce undergraduates to an extended, self directed project with an element of research to it. As someone who has supervised many such projects, I think that this is a pretty good one. Great work! At the same time, since the author is an undergraduate, presumably with a full coarse load, this is not a high quality research publication. The results are probably a bit shaky and the methodology is perhaps a bit off. But that's ok. Every excellent piece of work has to start somewhere.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#36

Post by panic »

bsdetector wrote: post #10

The author has a "lack of equal experience in the three evaluated languages", especially C, but is comparing them for development time and SLOC based on a sample size of one developer and one very small program. He rewrites a distributed program as a threaded one, because Rust and Go didn't have a distributed computing library, but says Go and Rust have "similar performance" in HPC. He says he was surprised at the C…

I agree that the paper isn't very convincing. One conclusion you might draw is that learning to write performant code in C is harder than learning to write performant code in Rust. That could just be a side-effect of Rust's much stronger standard library, though.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#37

Post by garthk »

ohquu wrote: post #22

Why are there so many bold terms and phrases throughout the text? It's extremely distracting.

I often find bold text helpful for key points, or for the first mention of key terms to help orient me in a document. There was something about the mix in this document that felt inconsistent. I found myself pausing to try to figure out why the author wanted my attention on this particular word or phrase.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#38

Post by semi-extrinsic »

ThePadawan wrote: post #7
semi-extrinsic wrote: post #5

Would've been nice with some internal links in that pdf, especially in the table-of-contents (links to sections and subsections). This is simple with hyperref.

Has the link has changed since your post? The ToC is linked for me.

Ah, sorry, it's just Quickoffice's pdf viewer (on Android) that somehow manages to not understand the links.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#39

Post by mrfloya »

dgacmu wrote: post #11

The problem with reading this in BS thesis form is that it hasn't gone through the wringer enough. Most of the performance comparison results vs C are bogus because (to its credit, as noted), the author accidentally compiled some important support libraries for the C version without optimization and used those on the cluster. Oopsie. As briefly mentioned in the previous chapter this performance regression might have…

Hey author of the thesis here.

You are correct this was definitely an oversight and I would have redone the measurements if time allowed for it. However I just want to state that the libaries were built with optimizations just not on the target platform which might not have been clear from the qoute.

Re: Evaluation of C, Go, and Rust in the HPC environment [pdf]

#40

Post by short_circut »

"In times where compilers are smart enough to basically rewrite and change code for performance reasons it is completely inexcusable that the order of source arguments to process is still that relevant."

After reading the comments on C and the difficulties the author had in the compilation process.... All I can say is Garbage in garbage out. I have worked in the HPC enviornment for a number of years using both Fortrash and C. The errors the author encountered would have been avoided by someone with any experience developing large softwares. I learned these things after working on my first project and now they barely, if at all, factor into my development time. Make sure you build and keep a dependency graph while you are developing and this becomes a non issue.

Post reply on HN