Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Landlock to Sandbox GNU Make

justine.lol

Post reply on HN
Print view View original

11–20 of 80 posts

Re: Using Landlock to Sandbox GNU Make

#11

Post by __float »

ghoward wrote: post #5

Good work, but I'm afraid that this will not really catch on because it's not that portable. I also think that Bazel has the same problem; most of the sandboxing works only on Linux. Until something like this works on Windows, it won't get wide adoption. And it is possible to do something like this on Windows. And personally, I would want to avoid the use of GNU make. Nevertheless, this is a step forward for those wh…

Bazel uses "sandbox-exec" on macOS, and has a more generic POSIX compatible sandbox if you can't use the Linux sandboxing tool too.

> And it _is_ possible to do something like this on Windows.

Are you able to point to any open source examples of this? Bazel supports Windows, and I'm sure they'd love to be able to support sandboxing on Windows as well.

Re: Using Landlock to Sandbox GNU Make

#13

Post by 3836293648 »

ghoward wrote: post #5

Good work, but I'm afraid that this will not really catch on because it's not that portable. I also think that Bazel has the same problem; most of the sandboxing works only on Linux. Until something like this works on Windows, it won't get wide adoption. And it is possible to do something like this on Windows. And personally, I would want to avoid the use of GNU make. Nevertheless, this is a step forward for those wh…

It is portable. That's the whole point of cosmopolitan binaries. Sure, there's probably a bunch of ugly hacks in cosmopolitan libc, but it works.

Re: Using Landlock to Sandbox GNU Make

#14

Post by brabel »

Very nice, this is almost exactly the kind of thing I wanted to have to create a higher level build tool (make files are IMO pretty terrible as a declarative language for building, but perhaps a good low level primitive to use to build systems like Bazel on top of). The only problem is the dependency on a fresh Linux feature (ported from OpenBSD if I understand correctly)... my ultimate goal was to have a Landlock thing that works on all OSs, but that may be really hard as not many languages abstract the file system away so that this can be implemented in the application-level (Dart has support for abstracting away the file system, and I was trying to use that, but it doesn't seem to support that kind of thing when running processes).

I suspect this should also make things like Nix and Guix easier... or maybe lighter-version of them easier as you don't need to implement the build sandox anymore when building software if you use Landlock Make.

Re: Using Landlock to Sandbox GNU Make

#15

Post by ghoward »

__float wrote: post #11
ghoward wrote: post #5

Good work, but I'm afraid that this will not really catch on because it's not that portable. I also think that Bazel has the same problem; most of the sandboxing works only on Linux. Until something like this works on Windows, it won't get wide adoption. And it is possible to do something like this on Windows. And personally, I would want to avoid the use of GNU make. Nevertheless, this is a step forward for those wh…

Bazel uses "sandbox-exec" on macOS, and has a more generic POSIX compatible sandbox if you can't use the Linux sandboxing tool too. > And it _is_ possible to do something like this on Windows. Are you able to point to any open source examples of this? Bazel supports Windows, and I'm sure they'd love to be able to support sandboxing on Windows as well.

It can be done on Windows, but not how Bazel would want to do it.

Basically, the sandbox has to be in the interpreter. And I guess I should be clear that it still won't be be as good as on Linux.

By the way, what's this "POSIX compatible sandbox"? chroot? That's not POSIX compatible.

Re: Using Landlock to Sandbox GNU Make

#16

Post by ghoward »

3836293648 wrote: post #13
ghoward wrote: post #5

Good work, but I'm afraid that this will not really catch on because it's not that portable. I also think that Bazel has the same problem; most of the sandboxing works only on Linux. Until something like this works on Windows, it won't get wide adoption. And it is possible to do something like this on Windows. And personally, I would want to avoid the use of GNU make. Nevertheless, this is a step forward for those wh…

It is portable. That's the whole point of cosmopolitan binaries. Sure, there's probably a bunch of ugly hacks in cosmopolitan libc, but it works.

Does Cosmopolitan's Landlocking work on anything besides Linux? By definition, that cannot be the case because Landlock is a Linux thing.

Cosmopolitan is portable, yes, but this sandboxing is not.

Re: Using Landlock to Sandbox GNU Make

#17

Post by gravypod »

jart wrote: post #8
gray_-_wolf wrote: post #6

This is definitely interesting and cool, however: > Landlock Make can build code five times faster than Bazel, while offering the same advantages in terms of safety. In other words, you get all the benefits of a big corporation build system, in a tiny lightweight binary that any indie developer can love. In terms of safety, maybe almost (bazel can check if the source files changed during the build, this (afaict) can…

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

> What I'd recommend for anyone really, is to just do what Google did. For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make. Then they switched to the huge scalable thing once they actually reached that inflection point.

Hopefully as a community we can build things that are scalable and as-simple-as Make. I think please.build is a step in the right direction but still too complicated.

Re: Using Landlock to Sandbox GNU Make

#18

Post by jart »

ghoward wrote: post #16
3836293648 wrote: post #13

Earlier quoted context omitted.

It is portable. That's the whole point of cosmopolitan binaries. Sure, there's probably a bunch of ugly hacks in cosmopolitan libc, but it works.

Does Cosmopolitan's Landlocking work on anything besides Linux? By definition, that cannot be the case because Landlock is a Linux thing. Cosmopolitan is portable, yes, but this sandboxing is not.

The Landlock Make binary runs on OpenBSD too, where it also supports sandboxing. But those are the only two platforms right now where it's fully-featured. As I mentioned in the article, the next logical step is to add FreeBSD support using jails. The other platforms, I'm not so sure about. Might have to do something like run the binary in a blinkenlights emulator to intercept the system calls of programs we didn't build ourselves. But it's kind of a moot point, since every platform supports spinning up a Linux thing that lets you build code. On Windows you have WSL. On Mac you have Docker. So I don't know why people are egging on with the portability angle. I mean, just having unveil() on Linux in addition to OpenBSD is HUGE! We've been waiting decades for this, and now it's finally here, thanks to Landlock!

Re: Using Landlock to Sandbox GNU Make

#19

Post by jeffreygoesto »

jart wrote: post #8
gray_-_wolf wrote: post #6

This is definitely interesting and cool, however: > Landlock Make can build code five times faster than Bazel, while offering the same advantages in terms of safety. In other words, you get all the benefits of a big corporation build system, in a tiny lightweight binary that any indie developer can love. In terms of safety, maybe almost (bazel can check if the source files changed during the build, this (afaict) can…

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

SCons [0] uses something similar (MD5 instead of SHA) since the end of the 90es, so at least that aspect is not a Google invention. The cache there is local though. We never had flaky builds with it and could extend it very nicely, unfortunately it was not very fast.

[0] https://en.m.wikipedia.org/wiki/SCons

Re: Using Landlock to Sandbox GNU Make

#20

Post by chubot »

jart wrote: post #8
gray_-_wolf wrote: post #6

This is definitely interesting and cool, however: > Landlock Make can build code five times faster than Bazel, while offering the same advantages in terms of safety. In other words, you get all the benefits of a big corporation build system, in a tiny lightweight binary that any indie developer can love. In terms of safety, maybe almost (bazel can check if the source files changed during the build, this (afaict) can…

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make

??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004.

So at most there were 6 years of using Make only, i.e. "google2".

Importantly, pre-Blaze google3 wasn't just GNU make -- Python was a huge part of it, which is why the Bazel build language Starlark looks like Python. It used to literally be Python, and now it's a Python-like language with parallel evaluation.

---

If you want to do what "scrappy Google" did these days, then you should use Python + Ninja. Ninja is meant to be generated, just like GNU Make was generated by Python. (A big difference is that GNU make has a big database of built-in rules that basically do nothing but slow down incremental rebuilds.)

I described that strategy a bit a few days ago: https://news.ycombinator.com/item?id=32307188

---

This work with Landlock looks very cool, and it would make a lot of sense for Ninja to have optional support for it. Some of the caveats are a bit scary but hopefully that can be worked out over time.

The way I was thinking of doing it was just to have a ./NINJA_config.py --slow-sandbox mode. So you can use any sandbox to warn you about missing dependencies, including something container-based like bubblewrap, symlink farms, or Landlock. I think that would work, though I haven't tried it. The shared library issue is tricky, etc.

It's very useful to have the build config / generator split for this reason, and many others (e.g. build variants go only in the first stage, not the second).

I wrote 3 substantial GNU makefiles from scratch and regretted it largely because it lacks this split -- it has a very tortured way of doing build "metaprogramming". IIRC one dimension of variants was OK, but 2 got you into the "write a Lisp in Make" territory. Might as well use Python (or Lua, etc.)

Post reply on HN