Live data from Hacker News

Linux from Scratch

linuxfromscratch.org

101–110 of 164 posts

Re: Linux from Scratch

#101
post #63

Between LFS and Stage 1 and 2 Gentoo installs back in the early 2000s during High School, this gave me a big leg up in my journey of learning about computers and Linux. I can't thank those project maintainers enough for helping me get my footing at such a young age and peaking my interest in computing. I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces…

Same, ricing Gentoo taught me a lot about linux and mainly how to fix it. 20+ year career later... I'm still learning how to fix it :)

Re: Linux from Scratch

#102
post #63

Between LFS and Stage 1 and 2 Gentoo installs back in the early 2000s during High School, this gave me a big leg up in my journey of learning about computers and Linux. I can't thank those project maintainers enough for helping me get my footing at such a young age and peaking my interest in computing. I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces…

Yep. Recompile all the things with different USE flags.

Recompiling the entire openoffice so it loads 1 second quicker.. just using a hacked up distcc cluster and a day's worth of CPU time. Bargain!

Re: Linux from Scratch

#103
post #77

This brings back memories. I built LFS on a Pentium based machine a few times and it was quite fun. I took a similar approach to building a custom bare metal system for a client that needed extra security. I sometimes wish that there was a well maintained bare bones distribution inspired by the LFS philosophy, that might help with managing complexity on the cloud. Scanning the thread, NixOS came up. I used to love Sl…

Alpine isn't good enough as a base system?

muslc isn't as fast as glibc (last time I checked), but I guess it's fine if you're ok with that.

Re: Linux from Scratch

#104
post #89

Earlier quoted context omitted.

NixOS separates packages. If the package foo contains a file /usr/bin/foo, NixOS installs it in /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. In order to make this separation work, Nix must sometimes rewrite binaries so that all references in the binary to /usr/bin/foo becomes references to /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. The advantage of this approach is that it gives more…

Extremely noob here, I was trying nixos and got real confused about how to install python packages as pip was not allowed at system level.

Nix purists would say that you should use flakes to declare all the dependencies for each project, and reference all Python dependencies as Nix packages there. Nix effectively tries to replace every package manager in existence, so all Python, Ruby, Emacs, etc. dependency trees are duplicated in Nix.

I think this is insane. Not only will many packages be missing from Nix, you will also have to wait for the upstream changes to actually propagate to Nix repositories. This all assumes, of course, that there are no packaging issues or incompatibilities in this repackaging.

This is one of the ways that Nix sometimes just gets in your way. I've been using Nix(OS) for several years now, and this still bothers me.

Instead of doing this, For Python specifically I would suggest installing pyenv, which Nix packages. Then enter a nix-shell with a derivation thingie[1,2], and install Python as usual with pyenv. Then you can use any Python version, and with pyenv-virtualenv (which Nix _doesn't_ package...), you can use venvs as you're used to. Sure, you don't get the benefits of the declarative approach and isolation as with "the Nix way", and you may run into other issues, but at least it's a workflow well known to Python devs. Hope it helps!

[1]: https://gist.github.com/imiric/3422258c4df9dacb4128ff94d31e9...

[2]: It took me way longer than I would like to admit to figure this out... This shouldn't be so difficult!

Re: Linux from Scratch

#105
Another project to check out if you enjoy LFS is buildroot.

Buildroot uses Kconfig (and associated tools like menuconfig, config, etc.) same as the kernel, to generate a configuration for building an embedded Linux system. This configuration can be committed and patched in the repo for your project. Adding an application or library automatically pulls in dependencies. Configs can be merged with fragments that add specific features.

Buildroot is capable of building its own cross compilation tool chain for many architectures, and enables developers to choose their own libc, and other toolchain configurations. It can also be configured to pull a prebuilt toolchain tarball to save time.

Packages are written in GNU Make, and macros are available for pulling code from GitHub and building packages using autoconf, cmake, meson, golang, Rust, python, and many more.

It works fantastically for anything from building an embedded Linux distro for something like digital signage, to a self-contained network bootable application for large scale automated provisioning of devices, to building minimal tarballs of filesystems to run as containers.

Re: Linux from Scratch

#106
post #83

Earlier quoted context omitted.

I grabbed a random page from the manual: https://www.linuxfromscratch.org/lfs/view/stable-systemd/cha... Every step is explained and every used parameter is documented.

I would say "explained". As a layman, who worked with linux previously (and even tried LFS a long time ago), but definitely don't have a deep understanding of linux, and eve my knowledge is not up-to-date: The sed command is not explained at all. Even the description is non-sense if you don't know already what they are talking about. What is this "default directory", why do you need to set it, why there and only ther…

Go deeper. You're right on the cusp of it since all of your questions are fantastic. Even digging into one of your questions will bring up highly relevant material for understanding how a Linux system functions.

    > The sed command is not explained at all. Even the description is non-sense if you don't know already what they are talking about. What is this "default directory", why do you need to set it, why there and only there, why that command works? Even the command itself is something, which I need to check the manual what the heck it does, because it's not a simple one.
By "default directory" they just mean that the upstream GCC source code has a file with default variables and they are modifying those variables with the sed command to use $prefix/lib and $prefix/usr/lib instead of $prefix/lib64 and $prefix/usr/lib64, e.g. lines that contain "m64=" and replacing "lib64" to be "lib". This is what sed is used for: To make string substitutions based on pattern matching. Think through ways of writing the sed command and testing on your own file to see how it behaves. This will lead you to more tools like diff, grep and awk.

    > > -enable-default-pie and --enable-default-ssp
    > The description is almost unusable. So we don't need it, but it's "cleaner", which in this context means exactly nothing. So what happens if I left out? Nothing? Then why should I care?
Go back and re-read the sections up to this point. Make note that you're in Chapter 5, which is the bootstrapping phase for the toolchain cross-compilers. Then look into the features that are mentioned. You can see descriptions by running the ./configure --help most times or looking up the GCC documentation. Those features are for security purposes and if you put that in perspective of the bootstrap phase they aren't needed if the only purpose of the temporary GCC binaries is to compile the final GCC in a later phase. To your point, perform an experiment and enable those features to see if there really is a difference other than time spent to compile. GCC incrementally adds security features like this and they are a big thing in hardened distributions.

    > > --disable-multilib
    > Okay, it doesn't support "something". I have no idea what is multilib, or why I should care. Basically the next arguments' description tells me that because it wouldn't work the compilation otherwise. And then..
A great feature to look up! Check out https://gcc.gnu.org/install/configure.html and search for the option and you'll find that it has to do with supporting a variety of target calling conventions. I can see how that'd be pretty confusing. It has to do with the underlying hardware support for application binary interfaces that GCC can utilize and it turns out you probably only need to support your native hardware (e.g. x86_64). That is, you're compiling your system from scratch and it'll only run on your native hardware (x86_64) but if you were a C/C++ programmer maybe you'd want to have support for other hardware (64-bit ARM systems are pretty common today as an example). So you can save time/space by disabling the defaults and honestly the defaults it includes are just not all that relevant on most systems today.

    > > --disable-threads, --disable-libatomic, --disable-libgomp, --disable-libquadmath, --disable-libssp, --disable-libvtv, --disable-libstdcxx
    > But why would they fail? I want to understand what's happening here, and I need to blindly trust the manual because they just tell me, that "they won't work, believe us".
Try it and find out. I would expect that they would fail due to reliance on other dependencies that may not have been installed or included in this bootstrapped build. Or maybe be/c those components don't behaved well with the LFS-based bootstrap methodology and ultimately aren't needed to bootstrap. Sure trust the LFSers but also think through a way to test your own assertions of the build process and try it out!

    > > --enable-languages=c,c++
    > Why are these the only languages which we need? What are the other languages?
GCC supports many language front-ends. See https://gcc.gnu.org/frontends.html. Only C/C++ is needed be/c you're bootstrapping only C and C++ based package sources. You can validate this as you build the remaining sections. It's conceivable that if you needed other languages in the bootstrap you could include them.

    > So at the end, descriptions are not really helping to understand what's happening, if you don't know already. The last time when I started LFS (about 10 years ago), that was my main problem. That you already need to know almost everything to understand what's really happening, and why, or reading manuals, or trying to find basically unsearchable information (like why libatomic compiling would fail at this step). So after a while, I started the blind copy-pasting, because I didn't have the patience of literary months, and when I realized that this was pointless, I gave up.
It's a steep learning curve, especially of a bootstrap which by its nature is circular! tbh, that's sort of the utility of LFS, it can take you up to a certain point but there are so many options and pitfalls in building a Linux system (or really any complex piece of software) and the real usefulness is pushing through, picking something and learning about it. Then using what you learned to apply to the unknown. GCC is one of the most important packages too, so there's a lot to unpack in understanding anything about it, but the impact is large.

Re: Linux from Scratch

#107

I remember trying to automate non-distribution builds of a functional GNU/Linux operating system and trying to not read Linux from Scratch but just official kernel.org documentation.[1] Unfortunately, the state of Linux documentation is so poor, you can't do it. You need to reference a number of third-party articles that kernel.org itself sometimes links you to. I believe kernel.org might also mention Linux from Scra…

> why you need particular dependencies straight out of initramfs. It came as a shock to me to learn that initramfs is mostly optional. It can be skipped, and you can boot straight into userland.

Yes, but if I understand correctly, you’d need to configure for this specifically. I think standard configurations and installations use them.

Re: Linux from Scratch

#108
post #77

This brings back memories. I built LFS on a Pentium based machine a few times and it was quite fun. I took a similar approach to building a custom bare metal system for a client that needed extra security. I sometimes wish that there was a well maintained bare bones distribution inspired by the LFS philosophy, that might help with managing complexity on the cloud. Scanning the thread, NixOS came up. I used to love Sl…

Alpine Linux is a very minimal base built around busybox and musl libc.

Void is another minimalist distro offering a choice of glibc or musl, and using runit as an init.

If you want more choices of init and a bleeding edge rolling release, Artix is a derivative of Arch with multiple init options in the core distribution.

Arch and Gentoo are the classics of do-it-yourself minimalism.

Re: Linux from Scratch

#109

Anyone else annoyed by “online books” sites like these where you have to play a convoluted game find-the-breadcrumbs from the website landing page to the beginning of the actual book content..

Two clicks from the landing page.

1. Choose the book you want to read (they call them sub-projects)

2. "Download" or "Read online"

Of course, they could reverse the nav order and provide first a "read online"/"download" and then let you pick which of the books you want to read, yielding ... two clicks to get to the one you want to read.

Re: Linux from Scratch

#110
I wonder what an equivalent "BSD from scratch" is like? Linux was assembled from a collection of parts while BSD is (reputably) a lot more 'designed from the ground up' Even a modern system like Fuchsia - what's that like to build from the ground up?

Or is it "You fool! Building your own kitbashed Gundam of an OS is the point."

Post reply on HN