Nix: Taming Unix with Functional Programming • Hacker News Board

Re: Nix: Taming Unix with Functional Programming

#161

Post by SuperSandro2000 » Sat, Aug 06, 2022, 3:24 PM UTC

infogulch wrote: post #113

SuperSandro2000 wrote: post #101

Earlier quoted context omitted.

The python ecosystem is not a giant mess, its just dependency hell. You need to work with what you get from upstream. > Nix actually integrates all package updates into one channel which nobody else anywhere does If you are using the python system packages then Arch, Debian and probably more are doing the same. > it's even worse because a lot of projects use python as a build dependency (?!) which then cascades these…

Thank you for your input! What do you think about the linked thread? > The python ecosystem is not a giant mess, its just dependency hell. There are dozens (?) of actively used package and environment management systems, with no consistency and no lockfiles, many packages ignore semver, and having multiple versions of a package installed causes weird issues. I'm not sure why "a giant mess" is an invalid descriptor, I…

> What do you think about the linked thread?

I already posted multiple posts in the thread under the same username. Other than adding important consumers of a dependency to its tests, the only solution I could come up with is to move AI/ML packages into their own repository where they can move at their own pace and do pins/overrides more freely and/or drastically reduce the amount of python packages in nixpkgs.

Re: Nix: Taming Unix with Functional Programming

#162

Post by SuperSandro2000 » Sat, Aug 06, 2022, 3:27 PM UTC

infogulch wrote: post #138

viraptor wrote: post #136

Earlier quoted context omitted.

Semver doesn't really apply here since nix always points at a specific version of each dependency. Multiple package versions are not visible to the same app. And you can include your own lockfile-like list. So in practice, it's not really a bigger problem for nix than for any other release. If you need to freeze versions for an app, you freeze them. If you don't or if you're packaging a library, you can (usually) rel…

The thing that you and sibling miss is that this only applies to the one direct dependency on a package, but not on dependencies required by that package. For that you have to work with any build and dependency resolution system that the package or its author uses, and you're up a creek without a paddle if the author doesn't care to help you debug your own system.

> that this only applies to the one direct dependency on a package, but not on dependencies required by that package

Right now in nixpkgs we manage all dependencies in all packages front to back. So all dependencies of a dependency and dependencies of dependency dependencies and so on.

Re: Nix: Taming Unix with Functional Programming

#163

Post by SuperSandro2000 » Sat, Aug 06, 2022, 3:31 PM UTC

zaphirplane wrote: post #129

jnxx wrote: post #76

Earlier quoted context omitted.

> As an observer it looks like Python's package ecosystem in particular is a giant mess. Most people would say it is. The better question is how to avoid such a mess. My take is that maintaining backward compatibility is a core principle which needs to be strictly observed to solve that problem. And yeah, that has also become a cultural issue with Python, as the Python2/3 breakage shows. So, could one sum it up in th…

Is the discussion here that Nix doesn’t support multiple side by side installations of different versions making it difficult to install two packages depending on the same package but different versions ? I thought in Nix you can have that, and python can have venv for independent library versions, is it that no one has done the work to combine the two ?

> I thought in Nix you can have that, and python can have venv for independent library versions, is it that no one has done the work to combine the two ?

A python environment is effectively a different take on a venv. Python packages is one giant python environment. For programs that life outside of python packages but use the packages from there we are free to apply overrides how we want. So it is possible to have different versions of a package in a different python environment.

Re: Nix: Taming Unix with Functional Programming

#164

Post by SuperSandro2000 » Sat, Aug 06, 2022, 3:34 PM UTC

jnxx wrote: post #150

SuperSandro2000 wrote: post #96

Earlier quoted context omitted.

Guix is also very verbose compared to nix.

Which is a good thing because most users do not want to configure and program Guix stuff all the time. Many will only use it every few months, and probably don't want to learn syntax and semantics again and again. Extreme terseness like APL or math notation is fine if you work with something all the time. However for infrastructure code and especially build systems, I think readability as well as robustness are much…

It's hard for me to find a good point about the verboseness and repetition of for example https://git.savannah.gnu.org/cgit/mediagoblin.git/tree/guix-...

Re: Nix: Taming Unix with Functional Programming

#165

Post by pdimitar » Sat, Aug 06, 2022, 4:28 PM UTC

rgoulter wrote: post #133

qudat wrote: post #111

Earlier quoted context omitted.

Relegated to a niche subgroup, I see. There's a significant difference between spending time learning a tool and sinking dozens and dozens of hours into a tool in order to perform tasks that are so basic for other package managers.

I don't think tasks which are basic for other package managers require dozens and dozens of hours sunk into nix to learn. "Niche subgroup" is about right in its current state. With text editors, VSCode is powerful and accessible to use, but there are power users who prefer to spend time learning Emacs. With package managers, nix is a power tool. It's not as accessible as it could be. But, the idea of "spend time lear…

Spending time to learn a tool is a standard requirement in our profession. Nobody but the laziest ones has a problem with it.

Programs like Nix, Emacs, VIM, Git -- they require a lot of time sunk into them to sometimes get even to basic productivity.

The latter is not okay. While I think it's unavoidable for Emacs and VIM, I've seen enough Nix and Git recipes and confusing command line aliases to conclude that Nix (and Git) can be much more friendly and have a smoother learning curve.

The ugly truth is that its community is not interested in that and even looks down on busy programmers who want to memorize a few shorthands and move on, which is a very valid mindset to have and I'm not okay with people looking down on it.

To me it looks like Nix is firmly headed in the direction of a yet another tool with a very good idea whose authors don't want to make it more usable and thus it remained a niche curiosity for people with too much free time... and the occasional corporate programming team that's perfectly served by its niche benefits.

I'd hate for Nix to become that. But at the moment everything points at this being its fate.

Re: Nix: Taming Unix with Functional Programming

#166

Post by lamontcg » Sat, Aug 06, 2022, 4:35 PM UTC

jnxx wrote: post #144

lamontcg wrote: post #105

Earlier quoted context omitted.

> My take is that maintaining backward compatibility is a core principle which needs to be strictly observed to solve that problem. Given that software developers never guess the correct design up front this means that you always have architecturally buggy software, and a bunch of complaining about why buggy-looking edge conditions are never fixed. There has to be some kind of release valve for software to evolve and…

> There has to be some kind of release valve for software to evolve and break backwards compatibility. You kinda insinuate that breaking backwards compatibility is kinda necessary at times. This is not the case. Projects like * the Linux kernel, or * the GNU C library, or * the Numeric -> Numpy transition around Python 2.0, or * Common Lisp (which is much older than Python) adopting Unicode are good examples that thi…

The problem with most open source software that is in package managers is that it is usually done by one person. It isn't started by someone with a decade of interface design, it is often their first large important project, they DONT do the well designed interfaces because they haven't made the mistakes in interface design yet which they'll eventually learn from. And then when it comes to backwards compatibility it is cheap for you to say they should just support their old interfaces forever, but that has a cost and creates more friction going forwards for the projected. When it is one person working on open source who isn't getting paid, that is all somewhat unreasonable to expect and you just won't ever get it. In your world what you'll wind up with instead of back compat breaking changes is just abandoned and rotting software as maintainers give up.

You could do this by arguing that languages need very thick and well-designed standard libraries, which means that hopefully there's a large business supporting the library and there are teams of reasonably well paid software engineers who are doing the design work up front for everything. You should be explicit about that though.

I'm kind of not surprised that you cite one of Linus' asshole rants to LKML as well. Try screaming that at a single-person open source maintainer and watch them decide it just isn't worth it any more and quit on the project entirely.

If you want that then don't use anything outside of your language's standard library and don't use package managers and contributed source code at all. Write everything else yourself, no dependencies, no worries about backwards compatibility breaks.

Re: Nix: Taming Unix with Functional Programming

#167

Post by rgoulter » Sat, Aug 06, 2022, 4:48 PM UTC

pdimitar wrote: post #165

rgoulter wrote: post #133

Earlier quoted context omitted.

I don't think tasks which are basic for other package managers require dozens and dozens of hours sunk into nix to learn. "Niche subgroup" is about right in its current state. With text editors, VSCode is powerful and accessible to use, but there are power users who prefer to spend time learning Emacs. With package managers, nix is a power tool. It's not as accessible as it could be. But, the idea of "spend time lear…

Spending time to learn a tool is a standard requirement in our profession. Nobody but the laziest ones has a problem with it. Programs like Nix, Emacs, VIM, Git -- they require a lot of time sunk into them to sometimes get even to basic productivity. The latter is not okay. While I think it's unavoidable for Emacs and VIM, I've seen enough Nix and Git recipes and confusing command line aliases to conclude that Nix (a…

> To me it looks like Nix is firmly headed in the direction of a yet another tool with a very good idea whose authors don't want to make it more usable and thus it remained a niche curiosity for people with too much free time.

What gives you the indication things are headed in the wrong way?

I think things are heading in the right direction.

The last year has seen nix flakes release to the stable nix version. Flakes are a big UX improvement to Nix.

The last few releases of nix have added improved support for debugging nix code. (Poor debugging UX was highlighted as a major pain point).

Efforts from major contributors are acknowledging the importance of improving documentation. - From the latest community survey, the steep learning curve and poor onboarding experience was noted as a major pain point. etc.

> even looks down on busy programmers who want to memorize a few shorthands and move on

Ehhh.

I don't think it's fair to say "vim is a bad tool because it requires learning to get used to it". -- Fortunately, developers aren't stuck between nano and vi, they've got highly accessible tools like VSCode.. or on the command line, even micro https://github.com/zyedidia/micro

Re: Nix: Taming Unix with Functional Programming

#168

Post by ParetoOptimal » Sat, Aug 06, 2022, 6:05 PM UTC

rgoulter wrote: post #134

baby wrote: post #128

Been using nix for a few weeks and while I feel like it really has some great ideas, here are what my pain points have been: - not great documentation, especially for newer features like flakes - nix wants to replace rustup when rustup is already doing great - nix doesn’t seem to work that well on mac. Not sure if it’s our config our that it’s painful on mac in general - the biggest issue: it doesn’t work well with t…

> the biggest issue: it doesn’t work well with tools (vscode, sublime merge, etc.) as you need to launch them within a nix shell and that doesnt work well (at least on mac). In what sense? In terms of "some nix shell provides some tools, and VSCode can't see those".. direnv is one way to work with this. e.g. direnv integrates with nix to integrate the nix shell at that path, and a direnv plugin for VSCode etc. can pi…

Vscode doesn't let you specify plugin load order and nix-env-selector needs to load before anything else or you can have problems last I checked.

Re: Nix: Taming Unix with Functional Programming

#169

Post by thomastjeffery » Sat, Aug 06, 2022, 6:14 PM UTC

rgoulter wrote: post #7

Neat article. For me, the key takeaways are: 1. 'Nix is to `tar -xf && make && make install` as C/C++ is to assembly'. In many ways, Nix applies the same kinds of improvements that other technologies have. 2. Nix does try and create an elegant programming model of Unix systems.. while the Nix programming language is pure, it interfaces with the Unix system by reading files and outputting files. I'm mixed on to what e…

Nix is an abstraction around `tar -xf && make && make install`. In fact, those commands are even executed by nix, just in a sterile reproducible environment.

What sets nix apart from other package managers is that you are never running `make install` on your root filesystem, but `make` can still dynamically link to libraries (that also aren't installed on the root filesystem) without editing the Makefile directly to find them.

This way, you can't break existing packages, you can trivially roll back changes (because updates are new instances), and you can always start over fresh.

The only problem is that you have to wrap every package in a derivation, then publish that derivation somewhere. Right now, all derivations are tracked in a single git repo (with dozens of branches), all coordinated over GitHub Issues, and referenced by nix itself by an arbitrary (versionless) name in a global namespace in this file: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/...

That last bit can be avoided by using pinning and flakes, but it's still the default way to use nixpkgs, and documentation doesn't clarify much or offer a better consistent UX paradigm.

Re: Nix: Taming Unix with Functional Programming

#170

Post by lamontcg » Sat, Aug 06, 2022, 6:18 PM UTC

btschaegg wrote: post #119

lamontcg wrote: post #105

Earlier quoted context omitted.

> My take is that maintaining backward compatibility is a core principle which needs to be strictly observed to solve that problem. Given that software developers never guess the correct design up front this means that you always have architecturally buggy software, and a bunch of complaining about why buggy-looking edge conditions are never fixed. There has to be some kind of release valve for software to evolve and…

I don't disagree, but it doesn't only have to go in this one direction. One of the most interesting things about Rust for example is how it tackles experimental implementations and has concepts in the compiler etc. that make unstable language features "first class". I'd say this will definitely yield better results than "well a couple of guys hacked around on some prototype forks of a compiler, and now we're stuck wi…

Rust has corporate sponsorship and a very experienced team of developers.

You won't get that level of attention to detail and commitment to getting the design right up front, and the willingness to maintain old APIs in the name of backwards compatibility in a single-person open source project published into a package manager done on someone's free time.

So you are probably arguing for very thick standard libraries which are maintained by the core language team, which is corporate sponsored, and a reduction in reliance on open source packages.

That also means as well that we shouldn't tolerate "shaming" of projects for taking a long time to fix and merge features since 95% of the work will be required to be done up front in thinking about the right shape of APIs.

I'm cool with all of that as long as the whole package comes along. The idea that a bunch of solo, unpaid open source maintainers are going to be doing good API design up front and maintaining perfect backcompat, while being incredibly responsive to PRs from the community is kind of "unicorn farts" levels of not going to happen in the real world. You sort of get what you pay for, and a bunch of unpaid solo volunteers are going to need to make breaking changes to fix their old mistakes and abandon maintaining their old tech debt. And if you paid nothing for it, really you're getting more than you deserve in that deal.