Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spack – scientific software package manager for supercomputers, Linux, and macOS

spack.io

Post reply on HN
Print view View original

81–90 of 107 posts

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#81

Post by timthorn »

mellosouls wrote: post #59
stonogo wrote: post #41

Earlier quoted context omitted.

Given that it's in use at Cambridge and several other HPC centers around the UK, I'm going to disagree. I think most people are cosmopolitan enough to set aside the UK's niche insult community to move forward with real work. I doubt there's any single-syllable morpheme which doesn't translate to something offensive somewhere.

the UK's niche insult community It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.

I'm UK native and had never heard of the word as an insult. I wonder if it's a regional thing?

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#82

Post by rekado »

arjvik wrote: post #68

Does this work with the "environment module" system (either the perl `module` or modern luamod `module`) commonly used in HPC, and what does the interop look like if so?

This may also be of interest: https://hpc.guix.info/blog/2022/05/back-to-the-future-module...

Guix also has a compatibility layer with environment modules, though you likely wouldn't compose environments that mix in Guix stuff with existing modules. It's intended as an output of Guix.

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#83

Post by mellosouls »

timthorn wrote: post #81
mellosouls wrote: post #59

Earlier quoted context omitted.

the UK's niche insult community It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.

I'm UK native and had never heard of the word as an insult. I wonder if it's a regional thing?

Perhaps a generational (and maybe class?) thing as well; it and related terms were vicious and influential enough to have motivated The Spastics Society to change their name to Scope in the mid-nineties [1]; I guess growing awareness of disability discrimination and bigotry also contributed to it being recognized as beyond the pale and disappearance from casual conversation with the corollary effect of some people outside those demographics being ignorant of how awful those terms were.

[1] https://www.bbc.com/news/blogs-ouch-26788607

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#84

Post by bowsamic »

timthorn wrote: post #81
mellosouls wrote: post #59

Earlier quoted context omitted.

the UK's niche insult community It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.

I'm UK native and had never heard of the word as an insult. I wonder if it's a regional thing?

Well the UK has done an amazing job at getting rid of almost all uses of the word "spastic" due to the campaign against it, so it could be for that reason

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#85

Post by boegel »

tgamblin wrote: post #29
JorgeGT wrote: post #6

How does it compares to other HPC software managers such as easybuild?

If I have to pick one difference, the fundamental one is that Spack does dependency resolution with a solver, while EB does not. So, in Spack, you write a package.py file (like say the one for gromacs: https://github.com/spack/spack/blob/develop/var/spack/repos/... ), and you can build that package with any of the versions/options/dependencies/etc. expressed in the package. With Easybuild, someone (maybe you?) has to…

EasyBuild lead developer here ^_^

Not all of this is 100% correct, so let me pitch in:

- EasyBuild currently doesn't have an uninstall option, that's true, but since every software installation sits in its own separate directory, it basically boils down to removing that directory + the environment module file that EasyBuild generated for it;

- EasyBuild can install "binary packages" (see the 'Binary' easyblock). Examples are the Intel compilers, CUDA, etc. We don't provides pre-built binaries for software that EasyBuild installs from source though, that's true;

- EasyBuild has no "environments" concept. The closest thing perhaps is the 'Bundle' easyblock, that can be used to "glue" different environment modules together. We mostly rely on the environment modules tool (Lmod) so this, see for example module collections: https://lmod.readthedocs.io/en/latest/010_user.html#user-col...;

- EasyBuild does indeed require an environment modules tool. Both Lmod (Lua-based) and Environment Modules (Tcl-based) are supported;

- EasyBuild also supports RPATH linking (see https://docs.easybuild.io/rpath-support/), but it's not the default.

> "EB is really set up to automate a certain type of installation common in HPC"

EasyBuild is definitely geared towards installing software on HPC systems, but there's no "certain type of installation common in HPC": we support software using a wide variety of installation procedures. But maybe you're referring to installing software in a shared filesystem (NFS, usually something like /apps).

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#86

Post by boegel »

throw0101b wrote: post #20
JorgeGT wrote: post #6

How does it compares to other HPC software managers such as easybuild?

From the 7th EasyBuild User Meeting (2022), 'A noob test: Spack "vs" EasyBuild' presentation is available on YouTube: * https://easybuild.io/eum22/#spack-vs-eb @tgamblin also presented 'The Spack 2022 Roadmap': * https://easybuild.io/eum22/#spack And will also present at 8th EUM (April): * https://easybuild.io/eum/#spack

Better link for Spack talk at EUM'23: (the 'eum' link is a moving target)

There's also an (now a bit outdated) talk compared EasyBuild with Spack and other alternatives like Nix/Guix which I gave a FOSDEM'18: https://archive.fosdem.org/2018/schedule/event/installing_so...

The EasyBuild User Meetings have always been very open to having talks on "competing" tools. Todd (or someone else) giving an update on recent developments in Spack is becoming a tradition (see also https://easybuild.io/eum21/#spack, https://github.com/easybuilders/easybuild/wiki/5th-EasyBuild..., etc.)

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#87

Post by boegel »

stabbles wrote: post #44
JorgeGT wrote: post #6

How does it compares to other HPC software managers such as easybuild?

Another point that I don't think gets emphasized a lot is that research software developers use spack as a dev tool on their laptop/cluster. Setting up easybuild locally is too much effort? If developers maintain spack recipes because they need them, it's much easier for sysadmins to deploy this software with spack too

I don't think it's about effort to set up EasyBuild (although we do have a hard requirement for a modules tool).

Spack is perhaps more attractive to software developers because it has specific features for that use, like the flexible dependency handling mechanism and the concretiser.

In my view, EasyBuild is better suited than Spack to maintain a central software stack, but I'm definitely biased. :)

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#89

Post by Y_Y »

stonogo wrote: post #41
trollied wrote: post #26

Earlier quoted context omitted.

This is definitely an international blunder. I’d say it’s more socially acceptable in the UK to call someone a c*t than it is to use this word. Can safely say it’ll get zero UK adoption due to the name. Shame.

Given that it's in use at Cambridge and several other HPC centers around the UK, I'm going to disagree. I think most people are cosmopolitan enough to set aside the UK's niche insult community to move forward with real work. I doubt there's any single-syllable morpheme which doesn't translate to something offensive somewhere.

"insult community"

Re: Spack – scientific software package manager for supercomputers, Linux, and macOS

#90

Post by tgamblin »

boegel wrote: post #85
tgamblin wrote: post #29

Earlier quoted context omitted.

If I have to pick one difference, the fundamental one is that Spack does dependency resolution with a solver, while EB does not. So, in Spack, you write a package.py file (like say the one for gromacs: https://github.com/spack/spack/blob/develop/var/spack/repos/... ), and you can build that package with any of the versions/options/dependencies/etc. expressed in the package. With Easybuild, someone (maybe you?) has to…

EasyBuild lead developer here ^_^ Not all of this is 100% correct, so let me pitch in: - EasyBuild currently doesn't have an uninstall option, that's true, but since every software installation sits in its own separate directory, it basically boils down to removing that directory + the environment module file that EasyBuild generated for it; - EasyBuild can install "binary packages" (see the 'Binary' easyblock). Exam…

Maybe I should clarify:

> it basically boils down to removing that directory + the environment module file that EasyBuild generated for it

Another important thing is knowing what depends on that package (so you don't remove something else's dependency). This is something that EB doesn't track after installation time.

> EasyBuild can install "binary packages"

What I meant here is that EB has no binary packaging system of its own. It has no binary package format, no signing, no way to take an installation and bundle it up into a file that can be installed (potentially in a different location) on another system. Spack, Nix, and Guix all have systems for creating and (re)installing binary substitutes for source builds (we call them build caches). EB can install someone else's binary package/distribution (we can too FWIW), but it can't create its own.

> But maybe you're referring to installing software in a shared filesystem (NFS, usually something like /apps).

Yes -- specifically that, with environment modules. Spack installations similarly all go in their own directories, but you can compose them in many different ways with environments and views. e.g., you can:

    - Create a symlink tree of many packages together in a common prefix (a view)
    - "project" package installations into different directory layouts with symlinks, hardlinks, or relocated copies
See https://spack.readthedocs.io/en/latest/environments.html#vie...

And you don't have to load packages one by one with modules. You can avoid modules entirely with `spack load` or use `spack env activate` to put everything in an env into your `PATH` (and other env vars).

Post reply on HN