Spack – scientific software package manager for supercomputers, Linux, and macOS • Hacker News Board

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#101

Post by eslaught » Tue, Mar 21, 2023, 4:26 PM UTC

trws wrote: post #94

eslaught wrote: post #78

I'm surprised that no one here has talked about the build times yet. As a new user, the first thing you notice out of the box is that even on a relatively beefy system, you spend ~2 hours getting to a working state, even when you're building a relatively modest set of packages. This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you'r…

> This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you're building already exists and works fine on the system. Try spack external find, and much of this problem goes away. It will register the existing packages as externals. As to the rest, spack actually has public binary mirrors with builds of many packages for common distributi…

> Try spack external find, and much of this problem goes away. It will register the existing packages as externals.

To be clear, this goes through a hard-coded list of packages, and adds them to Spack. It cuts maybe about 20-30 minutes off your 2+ hour build time. And it doesn't find any libraries (MPI being the most obvious one you'd want). So, better than nothing, but far from what it needs to be.

> public binary mirrors

Is that on by default? I went through this only a couple of months ago and I don't think any of my packages came from mirrors.

One practical challenge in using this in practice is that you definitely want some of your packages built with the local compilers, with maximally native settings (so native MPI, march=native, etc.). Other packages you may not care about at all (autoconf, etc.). So while I believe that Spack allows you to do that, it doesn't necessarily make it easy to do. I suspect when you tell Spack to build something with a certain compiler, it just builds everything from scratch because it can't tell where you actually care about getting the most tuned code.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#102

Post by eslaught » Tue, Mar 21, 2023, 4:34 PM UTC

EntrePrescott wrote: post #79

eslaught wrote: post #78

I'm surprised that no one here has talked about the build times yet. As a new user, the first thing you notice out of the box is that even on a relatively beefy system, you spend ~2 hours getting to a working state, even when you're building a relatively modest set of packages. This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you'r…

> like the lack of certain kinds of configuration isolation by default could you please explain a little what kind of "configuration isolation" you mean? > and the general difficulty of configuring certain kinds of things like what for example?

> could you please explain a little what kind of "configuration isolation" you mean?

It's been a while so I don't have the whole list off the top of my head, but there are several configuration files that live in $HOME by default unless you set SPACK_DISABLE_LOCAL_CONFIG=1. Those files can e.g., list compilers and software packages, if I recall, among various other settings.

> like what for example?

I was trying to follow someone's recipe for building something with Spack. I hit some sort of compiler error (sorry, it's been too long, don't remember). And the solution they suggested was to modify compilers.yml by hand to set various things. (Something about setting the right paths for using Cray's compiler wrappers with rocmcc...) That and I had to fuss with packages.yml to configure the system MPI. I remember trying to figure out how use "spack config set" to configure these files automatically, but the syntax was (a) completely undocumented and (b) not actually powerful enough in practice to configure either of those files. So I gave up on that, set SPACK_DISABLE_LOCAL_CONFIG and just saved the modified yaml files into my repository so that I can at least recover my settings if I need them in the future.

For advanced use cases, it seems like you basically need to modify these yaml files to get anything done, but the amount of documentation and support for doing this automatically is really lacking. (And the idea that you're supposed to require every user to do this manually is even more insane.)

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#103

Post by slavik81 » Tue, Mar 21, 2023, 4:57 PM UTC

eslaught wrote: post #101

trws wrote: post #94

Earlier quoted context omitted.

> This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you're building already exists and works fine on the system. Try spack external find, and much of this problem goes away. It will register the existing packages as externals. As to the rest, spack actually has public binary mirrors with builds of many packages for common distributi…

> Try spack external find, and much of this problem goes away. It will register the existing packages as externals. To be clear, this goes through a hard-coded list of packages, and adds them to Spack. It cuts maybe about 20-30 minutes off your 2+ hour build time. And it doesn't find any libraries (MPI being the most obvious one you'd want). So, better than nothing, but far from what it needs to be. > public binary m…

> And it doesn't find any libraries

There is some support for libraries with spack external find. For example, most of the ROCm libraries can be used that way.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#104

Post by telotortium » Tue, Mar 21, 2023, 7:08 PM UTC

alternatetwo wrote: post #95

miramba wrote: post #17

Earlier quoted context omitted.

Interesting, the same word with the same insulting meaning is used in northern Germany…TIL it‘s in English too. Maybe a leftover of the british occupation after WWII. Spacken Spacko Spack

Then there's "MongoDB", which is also slightly offensive in German, because Mongo refers to Mongoloid, a derogatory term for people with Downs syndrome.

Mongo has the same meaning in English, but as a Millennial, I've literally never heard this used in real life, only seen it referred to as an old insult. So it's quite likely that the founders and early users of MongoDB knew it was an insult.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#105

Post by soraminazuki » Wed, Mar 22, 2023, 11:40 AM UTC

tgamblin wrote: post #36

yjftsjthsd-h wrote: post #34

Earlier quoted context omitted.

Very neat:) Given the similarity in features, could you comment on how it compares to nix?

Main differences with Nix: * Spack has a dependency solver (concretizer). nixpkgs is managed by humans who do the dependency solving for you over time. What this means practically is that you can install 5 versions of some package with different dependencies and flag combinations with a few commands in Spack, while with Nix you would probably need to check out different commits of the nixpkgs repo and maybe do some h…

I'm not sure if I understand the first point. Doesn't Nix currently achieve the same thing feature-wise?

For example,

    spack install packageX %libA@2.0

would translate to

    nix-build -E 'with import  {}; packageX.override { libA = libAv2_0; }'

For this to work, both Spack and Nix would have to package libA v2.0. If it isn't, the user would need to create their own. Assuming a different version of libA is already packaged in Nix, here's how it would look like without the "nix-build -E" part:

    let
      pkgs = import  {};
      libAv2_0 = pkgs.libA.overrideAttrs (old: {
        src = pkgs.fetchzip {
          url = "...";
          sha256 = "...";
        };
      });
    in
    pkgs.packageX.override { libA = libAv2_0; }

I assume something similar would be required for Spack too.

So aside from Spack having a nicer shorthand syntax for customization, I don't get what Spack can do that Nix can't in terms of features. Or to be more specific, how a dependency resolver can eliminate human work.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#106

Post by tgamblin » Thu, Mar 23, 2023, 12:50 AM UTC

soraminazuki wrote: post #105

tgamblin wrote: post #36

Earlier quoted context omitted.

Main differences with Nix: * Spack has a dependency solver (concretizer). nixpkgs is managed by humans who do the dependency solving for you over time. What this means practically is that you can install 5 versions of some package with different dependencies and flag combinations with a few commands in Spack, while with Nix you would probably need to check out different commits of the nixpkgs repo and maybe do some h…

I'm not sure if I understand the first point. Doesn't Nix currently achieve the same thing feature-wise? For example, spack install packageX %libA@2.0 would translate to nix-build -E 'with import {}; packageX.override { libA = libAv2_0; }' For this to work, both Spack and Nix would have to package libA v2.0. If it isn't, the user would need to create their own. Assuming a different version of libA is already packaged…

Spack packages are parameterized, so there is one `package.py` file per package, not several, and you can have many versions and options declared. See, e.g., `zstd`: https://github.com/spack/spack/blob/develop/var/spack/repos/...

This was a conscious decision we made to allow many versions to be built without requiring split implementations and without checking out different commits -- that saves some work already. It also forces the repo maintainers to consider the other use cases, which tends (IMO) to make the recipes more portable.

But the real question is not just work saving but correctness. You cannot always just swap in libA@2.0 as in your example. For:

    packageX ^libA@2.0

Suppose that both packageX and libA depend on MPI (a versioned standard for which there are several implementations) and, further, libA requires MPI@3. Spack will ensure that:

    1. packageX and libA use the same MPI implementation (e.g., openmpi, mpich, mvapich)
    2. the MPI implementation chosen satisfies the provider requirement
    3. any version/option constraints that packageX has on MPI are satisfied along with those of libA.

You could also imagine that the two packages might have conflicts with certain implementations of MPI, e.g. say packageX conflicts with mpich and libA conflicts with mvapich. You'd have to choose mpich.

This isn't just an overlay; it's a constraint solve. The choice of libA@2.0 can have effects on other nodes in the graph, and different choices may need to be made elsewhere; maybe even things like disabling options or choosing different dependencies.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#107

Post by phissenschaft » Sat, Mar 25, 2023, 10:10 PM UTC

Been using Spack for a while to manage my machine learning package dependencies. It allows me to quickly spin up projects with complex dependencies (my current environment has 329 packages built ...). It's pretty easy to use with containers. It allows me to evaluate and migrate to different PyTorch/CUDA versions easily.