Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Landlock to Sandbox GNU Make

justine.lol

Post reply on HN
Print view View original

41–50 of 80 posts

Re: Using Landlock to Sandbox GNU Make

#41

Post by dima55 »

chubot wrote: post #20
jart wrote: post #8

Earlier quoted context omitted.

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make ??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004. So at most there were 6 years of using Make only, i.e. "google2". Importantly, p…

I don't disagree with anything you said, but I think it's important to point out that your experience is unique in that you're talking about build infrastructure for a massive codebase: most of Google's internal stuff, right?

In my experience plain GNU Make works great until you start working with massive projects. Similarly, the ninja speed improvement: you just don't see it unless you have a truly massive project, or you're using low-quality Makefiles, like what CMake produces. I've measured; I can't see it. I have many medium-sized projects, all with simple Makefiles, and it works really really well. I'm pretty convinced at this point that this should be the way to go for most projects out there.

Re: Using Landlock to Sandbox GNU Make

#42

Post by zvr »

jart wrote: post #8
gray_-_wolf wrote: post #6

This is definitely interesting and cool, however: > Landlock Make can build code five times faster than Bazel, while offering the same advantages in terms of safety. In other words, you get all the benefits of a big corporation build system, in a tiny lightweight binary that any indie developer can love. In terms of safety, maybe almost (bazel can check if the source files changed during the build, this (afaict) can…

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

(GNU) Make by default uses the file change timestamp to trigger actions. But this is definitely not the only way, and you can code your Makefile so that rebuilds happen when a file's checksum changes. IIRC, the GNU Make Book has the code ready for you to study...

Or, you might get more clever and say "when only a comment is changed, I don't want to rebuild"; file checksums are not the correct solution for this, so you can code another trigger.

Re: Using Landlock to Sandbox GNU Make

#44

Post by avar »

chubot wrote: post #20
jart wrote: post #8

Earlier quoted context omitted.

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make ??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004. So at most there were 6 years of using Make only, i.e. "google2". Importantly, p…

    I wrote 3 substantial GNU makefiles
    from scratch and regretted it largely
    because it lacks this split -- it has
    a very tortured way of doing build "metaprogramming".
GNU make definitely still has a lot of rough edges, but parts of your comment seems as though you're making claims about it that haven't been true in a decade or more.

For one, you don't need to "write a Lisp in Make", it has Guile built-in now (but probably didn't when you were actively using it): https://www.gnu.org/software/make/manual/html_node/Guile-Exa...

There's quite a few small things about modern GNU make that make it easier to use for meta-programming than it was even just a decade ago. It still has a lot of rough edges that I wish it didn't have though.

Re: Using Landlock to Sandbox GNU Make

#45

Post by jchw »

chubot wrote: post #20
jart wrote: post #8

Earlier quoted context omitted.

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make ??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004. So at most there were 6 years of using Make only, i.e. "google2". Importantly, p…

I will chime in in agreement with you, since the replies here are mostly to the contrary. Writing Makefiles by hand sucks eggs.

It's really worse than this because this probably assumed a specific Linux setup at least, but everything goes out the window once Windows is in the mix. If having to deal with potentially multiple shells was a problem with Make, having to deal with multiple shells on Windows (where it could be CMD, PowerShell, or a UNIX like shell under MSys or Cygwin...) is untenable.

Today the only obviously valid reason to use Make is because you have to or already are. Most Linux distros ship Ninja. Hell, MSVC ships Ninja.

There are many examples of Makefiles for open source projects that validate how ugly things can get without needing to be Google scale. One of the most elegant Makefile setups I can think of is Near's setup for bsnes/higan,and honest to goodness, it's still a pretty horrific mess.

I don't want to be hyperbolic, but also it's irresponsible for people to suggest you should just use Make. You shouldn't just use Make. Not even as a beginner. If you wanted a nice turnkey solution for a beginner, I'd suggest CMake or Meson, both of which happily generate Ninja files and also have the bonus of supporting Windows well. CMake has broad ecosystem support, despite it's many shortcomings. It's far from elegant, but definitely practical.

Re: Using Landlock to Sandbox GNU Make

#46

Post by jart »

chubot wrote: post #20
jart wrote: post #8

Earlier quoted context omitted.

One of the nice things about Bazel that the article didn't get a chance to go into is it uses SHA hashes of files, rather than file timestamps, to determine when an artifact has changed and therefore needs to be updated. It's slightly more costly to compute hashes, but it's necessary if you want to have something like a global cache of build artifacts, since synchronizing time across machines is hard. What I'd recomm…

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make ??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004. So at most there were 6 years of using Make only, i.e. "google2". Importantly, p…

I wouldn't say it's scary. There's always been full transparency with the caveats and they're being lifted incrementally. https://twitter.com/l0kod/status/1556378406983458818 The workarounds are perfectly reasonable. Also take into consideration that Landlock is so simple as a security tool, that it really opens itself up to people like us being able to focus in on the opportunities for improvement. A lot of security stuff, like containers, is so byzantine that no one would enjoy having open discussions about their design. Landlock has felt different to me, simply because we are having these conversations, and it's been so engaging that it really makes me want to believe in Torvald's whole mythology about the many eyeballs.

Email me if there's anything I can do to help Ninja adopt Landlock. The Cosmopolitan Libc pledge() and unveil() implementations are written to be relatively easy to transplant into other codebases. I'd love to see a broader audience benefiting from our work.

Re: Using Landlock to Sandbox GNU Make

#47

Post by jart »

jchw wrote: post #45
chubot wrote: post #20

Earlier quoted context omitted.

For the first decade and a half of Google's company lifecycle, they got along just fine with GNU Make ??? Google was started in 1998, and Bazel was created ~2006 as a replacement for Python + GNU Make ("gconfig"). I was on that team, though I only worked on Blaze a tiny bit. The "google3" build migration was sometime around 2003 or 2004. So at most there were 6 years of using Make only, i.e. "google2". Importantly, p…

I will chime in in agreement with you, since the replies here are mostly to the contrary. Writing Makefiles by hand sucks eggs. It's really worse than this because this probably assumed a specific Linux setup at least, but everything goes out the window once Windows is in the mix. If having to deal with potentially multiple shells was a problem with Make, having to deal with multiple shells on Windows (where it could…

That's Windows' problem and it's not even a real problem anymore because make runs fine in WSL. Microsoft has pretty much gotten their act together in the last four years in supporting open developer tools. They've got bash and ANSI support. It's great. Give them credit where credit is due. It's time to say goodbye to shoehorning unix devops into win32. Doing that gets sillier each year. Especially since, as Microsoft has been generous in supporting tools, they've certainly been boiling the frog with their win32 environment. The last time I extracted a zip file containing dev work on Windows, it extracted at 64 kilobytes per second to the local hard drive, because their virus technology was scanning and uploading everything I was doing. How can you build software in that kind of environment? And why is it that it's always the voices coming from our own community who insist that we should. People shouldn't fall that far in love with an operating system because even with perfect support, builds are still going to go slow on Windows. Use WSL.

Re: Using Landlock to Sandbox GNU Make

#48

Post by dwheeler »

jart wrote: post #4
djhaskin987 wrote: post #3

I wonder why there was no mention of simply contributing these 200 lines to GNU make proper instead of forking. Super cool though.

That 200 lines doesn't include our pledge() implementation, which Make assumes is provided by the C library. Right now only Cosmopolitan Libc and OpenBSD have an unveil() implementation. It would take some thought to decide what the best approach would be for incorporating something like that into GNU Make. So I'm waiting on more feedback from the community and the GNU developers. Because upstreaming is totally somet…

I'd like to see this upstream. It's okay if the functionality only works on some platforms... it would force fixing of makefiles, and all platforms would benefit from that result.

Re: Using Landlock to Sandbox GNU Make

#50

Post by shakna »

spockz wrote: post #40

The only thing I’m really missing for my usecase with make is the ability to specify goals/targets other than files. E.g., I want a docker image to be present in the daemon. Basically, calling out to a function. (Probably somebody is going to correct me and say there is something there in the form of conditionals or something..)

GNU Make has Guile built-in, so you've got the whole of Scheme (the language), if you want to do something a bit more complicated.
Post reply on HN