Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spack – scientific software package manager for supercomputers, Linux, and macOS

spack.io

Post reply on HN
Print view View original

51–60 of 107 posts

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#51

Post by wdconinc »

ephimetheus wrote: post #45

I’ve tried to like this more than once, but once you hit some cryptic error somewhere in the build you’re not going to have a great time. Also unfortunately it doesn’t always play nicely with lots of high energy physics software which really leans on using LD_LIBRARY_PATH and running custom executables during builds. I’ve had bad experiences with ROOT and Gaudi with no obvious paths forward for fixing things.

We have used spack to maintain our 1000-person user environments at the electron-ion collider (including ROOT, Gaudi) for the past two years and couldn't have done it otherwise.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#52

Post by prpl »

ephimetheus wrote: post #45

I’ve tried to like this more than once, but once you hit some cryptic error somewhere in the build you’re not going to have a great time. Also unfortunately it doesn’t always play nicely with lots of high energy physics software which really leans on using LD_LIBRARY_PATH and running custom executables during builds. I’ve had bad experiences with ROOT and Gaudi with no obvious paths forward for fixing things.

Both of those are non-trivial and annoying to say the least, I know ROOT has a few very smart people managing build recipes in conda-forge too.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#53

Post by prpl »

I was excited for Spack and used it for some small packages, but it didn’t have a proper binary artifacts then. Ended up settling on conda-forge and it’s okay.

If I was doing a large scientific computing project today, I’d probably reach for bazel with some conda rules hooked up to conda-forge to set up third party packages.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#54

Post by wetbaby »

trollied wrote: post #26
kichimi wrote: post #4

An unfortunate name, a la Nonce Finance and Git. https://en.m.wiktionary.org/wiki/spack

This is definitely an international blunder. I’d say it’s more socially acceptable in the UK to call someone a c*t than it is to use this word. Can safely say it’ll get zero UK adoption due to the name. Shame.

Yeah, spack(er) is really up there for an insult. You really have to be extremely vitriolic to call someone a spack.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#55

Post by tgamblin »

331c8c71 wrote: post #50
tgamblin wrote: post #40

Earlier quoted context omitted.

The Spack DSL for defining versions, options, etc. is declarative. That part of a package exposes options to the solver and very much allows reuse, in the sense that the same package can be built many different ways, and even with different dependencies. The whole system is parameterized in ways that Nix is not. The imperative part of a Spack package is the build recipe, which AFAICT is not so different from bundling…

Thanks for the answer. > That part of a package exposes options ... in the sense that the same package can be built many different ways, and even with different dependencies. It is quite a common pattern in nix to have package build flags and the dependencies as inputs in a derivation. This definitely makes it possible to build in different ways and with different dependencies.

Yep! I'm not a nix expert but my understanding is that this is typically done with overlays. So you can, say, swap out openmpi for mpich, but mpich requires a different version of hwloc that conflicts with the one defined in some other branch of your derivation, you're out of luck. In Spack that is handled through dependency resolution. There have been discussions of how to deal with this in Nix, e.g.: https://discourse.nixos.org/t/concept-use-any-package-versio....

Similarly, for packages like HDF5 that tend to be depended on with options that affect its API (+parallel, ~parallel, etc.), I do not think there are automatic ways to get nix to ensure that your request for hdf5+parallel is consistent with other packages in the graph.

Possibly worth a mention: for uarch flags like the ones shown here: https://nixos.wiki/wiki/Build_flags, the user is responsible for setting them specifically for the requested compiler. We've abstracted that with `archspec` (https://github.com/archspec/archspec) so that you can just ask for a target and the correct compiler flags are injected. e.g., `target=cascadelake` or `target=zen3`. So if you switch compiler, you do not have to look up these details. See https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pd... for more on how it works.

The salient parts of archspec aren't in a python library but in a JSON file (https://github.com/archspec/archspec-json/blob/master/cpu/mi...) so it's something that could probably be incorporated into Nix.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#57

Post by mellosouls »

Moissanite wrote: post #31
kichimi wrote: post #4

An unfortunate name, a la Nonce Finance and Git. https://en.m.wiktionary.org/wiki/spack

For Americans who don't want to click the link: "Spack" is basically a milder British-English version of "retard". Imagine how well the "Spaz programming language" would do in your market.

It's certainly not milder.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#59

Post by mellosouls »

stonogo wrote: post #41
trollied wrote: post #26

Earlier quoted context omitted.

This is definitely an international blunder. I’d say it’s more socially acceptable in the UK to call someone a c*t than it is to use this word. Can safely say it’ll get zero UK adoption due to the name. Shame.

Given that it's in use at Cambridge and several other HPC centers around the UK, I'm going to disagree. I think most people are cosmopolitan enough to set aside the UK's niche insult community to move forward with real work. I doubt there's any single-syllable morpheme which doesn't translate to something offensive somewhere.

the UK's niche insult community

It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.

Post reply on HN