Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Landlock to Sandbox GNU Make

justine.lol

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Using Landlock to Sandbox GNU Make

#51

Post by avar »

spockz wrote: post #40

The only thing I’m really missing for my usecase with make is the ability to specify goals/targets other than files. E.g., I want a docker image to be present in the daemon. Basically, calling out to a function. (Probably somebody is going to correct me and say there is something there in the form of conditionals or something..)

That's easy to do, but if you want to make it compatible with incremental builds the best way is to create a "I did it already" empty file as a dependency marker. E.g. "build" a ". build/called/apt-install-deps" or whatever.

Re: Using Landlock to Sandbox GNU Make

#52

Post by colatkinson »

ghoward wrote: post #27
__float wrote: post #23

Earlier quoted context omitted.

What interpreter? Why do you assert that it's "not how Bazel would want to do it"? No, it's not chroot -- it uses a tree of symlinks, created specifically for each process being run, that assumes the process isn't trying to escape the 'sandbox' too much. At the end of the day, a _strong_ sandbox is going to require some OS-specific support, and this could be seen as a kind of graceful degradation when there's not a b…

Why do my comments seem combative? All I said was basically good job, but it will unfortunately not be used much. Why is that combative? With regards to chroot, I stand corrected. I knew it was a tree of symlinks, but I thought it was also more than that because symlinks alone don't seem like a sandbox. Honestly, Cosmopolitan's system appears to be more of a sandbox than that. I agree that a strong sandbox is going t…

> With regards to chroot, I stand corrected. I knew it was a tree of symlinks, but I thought it was also more than that because symlinks alone don't seem like a sandbox. Honestly, Cosmopolitan's system appears to be more of a sandbox than that.

To be totally clear: the tree of symlinks thing is a fallback, used only when lacking platform support or when sandboxing is explicitly turned off [0]. On Linux, the normal sandboxing strategy is to use namespaces, like most container runtimes. On Mac it apparently uses sandbox-exec (some opaque Apple tool), as was mentioned above. Chroot, being both non-POSIX, requiring root access on many systems, and not providing the necessary facilities is not really a great fit -- which I assume is why it's not used.

There was experimental Windows sandbox support at one point [1] based on how MS does it for BuildXL (their own build tool for giant monorepos) [2]. Unfortunately it doesn't seem to be maintained, and under the hood it's kinda ugly -- it actively rewrites code in-memory to intercept calls to the Win32 APIs [3], which was apparently the cleanest/best way MS could come up with. However, from Bazel's POV it works in a roughly similar way -- you spawn subprocesses under a supervisor, which is in charge of spinning up whatever the target process is with restrictions on time/memory usage/file access.

On the "sandbox in the interpreter" thing: what kind of checks are you envisioning? It seems like putting checks at that level would end up leaving a lot out -- the goal of any build system is to eventually spawn an arbitrary process (Python, gcc, javac, some shell script, etc.) and so even with extensive checks in starlark you'd end up with accidental sandbox breaks all over the place. For pure starlark rules you could e.g. check that there are no inputs from /usr, but even then if gcc does it implicitly, you're SOL. Or am I thinking of the wrong kind of checks?

EDIT: somehow missed your sibling comment. Nix is definitely cool, and is pretty similar to how Bazel does things with regards to explicit build graphs. The check for "well-known commands" would also be cool IMO. That said, Nix also has a chroot-y sandbox-y thing it uses to spawn processes -- so they're not all that different [4].

[0] https://bazel.build/docs/sandboxing#sandboxing-strategies

[1] https://github.com/bazelbuild/bazel/issues/5136#issuecomment...

[2] https://github.com/microsoft/BuildXL/blob/master/Documentati...

[3] https://github.com/microsoft/Detours/wiki

[4] https://nixos.wiki/wiki/Nix_package_manager#Sandboxing

Re: Using Landlock to Sandbox GNU Make

#54

Post by enriquto »

spockz wrote: post #40

The only thing I’m really missing for my usecase with make is the ability to specify goals/targets other than files. E.g., I want a docker image to be present in the daemon. Basically, calling out to a function. (Probably somebody is going to correct me and say there is something there in the form of conditionals or something..)

> E.g., I want a docker image to be present in the daemon.

This sounds more like a limitation of docker than of make. I don't use docker, but how on earth something called a "docker image" is not a file?

Re: Using Landlock to Sandbox GNU Make

#55

Post by jchw »

jart wrote: post #47
jchw wrote: post #45

Earlier quoted context omitted.

I will chime in in agreement with you, since the replies here are mostly to the contrary. Writing Makefiles by hand sucks eggs. It's really worse than this because this probably assumed a specific Linux setup at least, but everything goes out the window once Windows is in the mix. If having to deal with potentially multiple shells was a problem with Make, having to deal with multiple shells on Windows (where it could…

That's Windows' problem and it's not even a real problem anymore because make runs fine in WSL. Microsoft has pretty much gotten their act together in the last four years in supporting open developer tools. They've got bash and ANSI support. It's great. Give them credit where credit is due. It's time to say goodbye to shoehorning unix devops into win32. Doing that gets sillier each year. Especially since, as Microsof…

You can't really use WSL to develop or use native Windows software. WSL is good for when your target is Linux and otherwise not applicable.

It's also not Window's problem. It's yours, when you try to develop software and have Windows users.

Re: Using Landlock to Sandbox GNU Make

#56

Post by thayne »

gravypod wrote: post #35
chubot wrote: post #21

Earlier quoted context omitted.

Eh Make is not good for most problems -- see experiences here by dwheeler, me, frankohn: https://news.ycombinator.com/item?id=32301606 And this other thread I linked there, with feedback from FreeBSD engineers: https://lobste.rs/s/7svvkz/using_bsd_make#c_bfwcyc --- Ninja basically gives you the parts of GNU make that are good, without the cruft and slowness. And you can learn it in 20 minutes, unlike GNU make

(Opinions are my own) > Eh Make is not good for most problems Agreed. I spend a lot of time advocating for Bazel for this reason. > Ninja basically gives you the parts of GNU make that are good, without the cruft and slowness. And you can learn it in 20 minutes, unlike GNU make Ninja seems more like a side-grade than a wholesale improvement. It does not fix all the problems that need to be addressed. Compare the two:…

> I spend a lot of time advocating for Bazel for this reason.

Bazel works pretty well for large scale projects. There are definitely things I don't like about, but I would agree that for large projects it is better than make.

But for small to medium size projects, bazel adds a lot of complexity and has the problems you mentioned, for not that much benefit. Especially if you are using a language that doesn't have built in support in bazel,or do something that doesn't match up with bazel's way of doing things.

Re: Using Landlock to Sandbox GNU Make

#57

Post by ghoward »

colatkinson wrote: post #52
ghoward wrote: post #27

Earlier quoted context omitted.

Why do my comments seem combative? All I said was basically good job, but it will unfortunately not be used much. Why is that combative? With regards to chroot, I stand corrected. I knew it was a tree of symlinks, but I thought it was also more than that because symlinks alone don't seem like a sandbox. Honestly, Cosmopolitan's system appears to be more of a sandbox than that. I agree that a strong sandbox is going t…

> With regards to chroot, I stand corrected. I knew it was a tree of symlinks, but I thought it was also more than that because symlinks alone don't seem like a sandbox. Honestly, Cosmopolitan's system appears to be more of a sandbox than that. To be totally clear: the tree of symlinks thing is a fallback, used only when lacking platform support or when sandboxing is explicitly turned off [0]. On Linux, the normal sa…

Yes, I agree that Bazel and Nix are not much different. Nix seems to be even more sandbox-like than Bazel, and that's good in my opinion.

Beyond what they do, I'd like checks that are even more invasive, more cautious about letting the build script do anything.

For example, if you're on Linux, a bad actor build script could technically mount the root directory `/` underneath the sandbox area in //rootdir/` using Linux's bind mounts feature and then `rm -rf //rootdir/`. Whatever it has permission to delete will be deleted (unless I'm unaware of some safety feature in bind mounts that prevents this besides needing root).

I would like checks that restrict a build to just performing those actions necessary to the build. You could, for example, have a permission policy, say for a particular package that you don't trust, that only allows that package to spawn GCC and the linker. If that package goes rogue in its build script, it would be stopped dead the first time it tried to either use `rm` or use a bind mount.

That's the sort of checks I'm referring to: checks for fine-grained permissions on what a build can do.

My idea is to take that even further and make it possible to have those checks in software that you compile from source so that you can stop the software from going rogue too. How I am going to do that, I'll leave unsaid for now, but I'm working on it.

Re: Using Landlock to Sandbox GNU Make

#58

Post by thayne »

chubot wrote: post #21
gravypod wrote: post #17

Earlier quoted context omitted.

> What I'd recommend for anyone really, is to just do what Google did. For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make. Then they switched to the huge scalable thing once they actually reached that inflection point. Hopefully as a community we can build things that are scalable and as-simple-as Make. I think please.build is a step in the right direction but still…

Eh Make is not good for most problems -- see experiences here by dwheeler, me, frankohn: https://news.ycombinator.com/item?id=32301606 And this other thread I linked there, with feedback from FreeBSD engineers: https://lobste.rs/s/7svvkz/using_bsd_make#c_bfwcyc --- Ninja basically gives you the parts of GNU make that are good, without the cruft and slowness. And you can learn it in 20 minutes, unlike GNU make

Ninja is self described as an assembler for builds. You aren't expected to write ninja build files manually.

Re: Using Landlock to Sandbox GNU Make

#59

Post by benreesman »

Others have said similar things, so my point is mostly one of emphasis. Bazel goes deep on finding edges in the build graph.

Example: C/C++ has this preprocessor step where ‘#include’ and stuff happen. It’s very cheap compared to what comes next.

If you change white space or comments or whatever, the preprocessor runs, hashes the same, now the .o is good, now the link is good.

OP’s stuff is cool, but just wanted to make sure everyone is clear that industrial strength build systems like Bazel take on that constant factor intentionally. They are designed for building big, complex, polyglot code bases.

Re: Using Landlock to Sandbox GNU Make

#60

Post by Cloudef »

jchw wrote: post #55
jart wrote: post #47

Earlier quoted context omitted.

That's Windows' problem and it's not even a real problem anymore because make runs fine in WSL. Microsoft has pretty much gotten their act together in the last four years in supporting open developer tools. They've got bash and ANSI support. It's great. Give them credit where credit is due. It's time to say goodbye to shoehorning unix devops into win32. Doing that gets sillier each year. Especially since, as Microsof…

You can't really use WSL to develop or use native Windows software. WSL is good for when your target is Linux and otherwise not applicable. It's also not Window's problem. It's yours, when you try to develop software and have Windows users.

Mingw exists
Post reply on HN