Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is it time for a new Embedded Linux build system?

yoebuild.org

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Is it time for a new Embedded Linux build system?

#71

Post by palata »

lpribis wrote: post #64
palata wrote: post #61

My experience is that Yocto is a bit like CMake: people hate the mess they make with it and blame the system, even though they never bothered learning to use it. When someone starts with "why cross-compiling the system if it's bearable to compile it directly on the device?", it really sounds like they are just not comfortable with cross-compilation. To me it would be similar to say "why would you need a compiled lang…

> My experience is that Yocto is a bit like CMake: people hate the mess they make with it and blame the system, even though they never bothered learning to use it. I agree this is true a lot of the time, speaking as someone who has made many Yocto messes and later regretted. But you also can't discount all of the messes made by silicon OEMs that you are forced to deal with to use their SoCs. E.g. I recently did a pro…

> But you also can't discount all of the messes made by silicon OEMs that you are forced to deal with to use their SoCs.

Yes, totally! It's a bit sad because we don't usually have the luxury to select the hardware based on the quality of the BSP. First because they all seem to be terrible, and second because it is somehow "hidden" cost. One selects the hardware based on its price and capabilities, and the software part is left as an exercise.

But as you say it's not a problem of Yocto, it's a problem of the OEMs being in a position to make a mess because it doesn't have an impact on them. A bit like Apple is terrible for developers (in my experience) because anyway developers don't have a choice.

Re: Is it time for a new Embedded Linux build system?

#72

Post by palata »

actionfromafar wrote: post #62
palata wrote: post #61

My experience is that Yocto is a bit like CMake: people hate the mess they make with it and blame the system, even though they never bothered learning to use it. When someone starts with "why cross-compiling the system if it's bearable to compile it directly on the device?", it really sounds like they are just not comfortable with cross-compilation. To me it would be similar to say "why would you need a compiled lang…

CMake at least used to be like this: a black plastic bag of LEGOs, you empty it on the floor, there are some instructions there, but some of the images don't look like any of the pieces you see on the floor. Also there are some random rusty bolts, lint, and pieces from an Erector Set. There's an empty tape roll, someone wrote "buy more, ask forum" with a sharpie on it. You can make great things with that, but it was…

I get the feeling, it is a... "different" language. But I never understood the criticism about the documentation. At least in the last decade, I have always found the CMake documentation pretty good. Everything is documented.

Probably CMake is not a good fit for very, very big projects. But the vast majority of projects are not that big.

Usually what I see with CMake is self-inflicted pain, often where the devs start adding custom functions and mix it with python and stuff. If you keep it simple, it's very clear and does the job with surprisingly few keywords, I find. And I have setup tens of projects with CMake.

Re: Is it time for a new Embedded Linux build system?

#73

Post by ValdikSS »

Wow, no cross-compilation as a feature?!

Architecture emulation with qemu-userspace is 5-10x slower than running a native build.

The only 'proper' no-friction cross-compilation system I know is Debian/Ubuntu, because it allows to install foreign architecture libraries into your native system, and has all the wrappers for cross-compilation in dpkg-buildpackage.

This way you don't have to maintain cross-compiler or toolchain/sysroot, you just install whatever dependencies you need with the regular `apt install` of another architecture.

I'd like to have more friendly tooling around that instead of a slow compilation of architecture emulation.

Re: Is it time for a new Embedded Linux build system?

#74

Post by superxpro12 »

KingMachiavelli wrote: post #17

I've been using Nix for "embedded" systems for few years now and it works fine. I don't quite understand why "embedded" has historically meant learning a completely separate tool. I've been building for x86 and ARM servers for 5+ years now, so why should targeting an ARM board be any different? Like the article points out, the software stacks people use for embedded devices are the same as people use non-embedded use…

Hey, you completely forgot about embedded devices that dont run linux. It's not all linux ya know

Re: Is it time for a new Embedded Linux build system?

#75

Post by structural »

okanat wrote: post #26
KingMachiavelli wrote: post #18

Earlier quoted context omitted.

Embedded just means ARM 99% of the time and it's cheaper and easier to use native ARM servers (AWS has them cheap) than to make 100% of software cross compile. Some parts of the firmware might need to be cross compiled but those projects are designed to cross compile.

Your target build environment often differs significantly from your server environment, so you end up needing a different toolchain and all of the problems that come with cross-compilation anyway. The toolchain and ABI settings that produce small, battery or instruction cache efficient are usually not you want on servers.

Run on server, chroot into target environment, configure compiler flags for eventual target, done. It's really just an afternoon of setup and then the problem is solved.

Re: Is it time for a new Embedded Linux build system?

#76

Post by finnthehuman »

nrdvana wrote: post #43
drdexebtjl wrote: post #25

Earlier quoted context omitted.

>native builds This is the complete opposite way, actually. We need cross-compiling that is just as effortless as native compilation. You should be able to build complex software on a powerful computer and perform costly optimization, then run it on a low-powered device.

It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI, then start a docker container and assemble a system from Alpine's package repo, then compile a kernel and a few in-house things, then extract a subset of that into an image. No cross-compile, and most of the useful software is in pre-built binary packages with the compile-time opt…

> It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI

I’m already sitting at an expensive computer, if everything else were equal I’d rather cross compile than buy more hardware.

I’m open to reasons that’s presently too hard of a workflow to enable so I’m (literally) paying for a penalty, but I don’t see how it’s a better idea.

Re: Is it time for a new Embedded Linux build system?

#77

Post by KingMachiavelli »

throwawayqqq11 wrote: post #60
KingMachiavelli wrote: post #20

Earlier quoted context omitted.

What do you mean by image? In Nix you instantiate a NixOS system and specify the architecture. There are a number of tools to build a disk image from a NixOS system. $ my-arm-system = nixpkgs.lib.nixosSystem { system = "aarch64-linux"; modules = [ ./configuration.nix ]; }; $ :b arm-system.config.system.build.images.iso

PSA: nixos-rebuild build-vm (--flake DIR#HOSTNAME / configuration.nix) ./result/bin/SCRIPT to build and launch a nixos vm with qemu.

IMO the nixos-rebuild scripts are unnecessary and just obscures what’s being built.

build-vm is the same as building the configs “config.system.build.vm” attribute.

nixos-rebuild is fine for getting started but you end up needing to use the attribute path to do anything more complicated like CI caching or inspecting the drvPath.

Re: Is it time for a new Embedded Linux build system?

#78

Post by KingMachiavelli »

okanat wrote: post #26
KingMachiavelli wrote: post #18

Earlier quoted context omitted.

Embedded just means ARM 99% of the time and it's cheaper and easier to use native ARM servers (AWS has them cheap) than to make 100% of software cross compile. Some parts of the firmware might need to be cross compiled but those projects are designed to cross compile.

Your target build environment often differs significantly from your server environment, so you end up needing a different toolchain and all of the problems that come with cross-compilation anyway. The toolchain and ABI settings that produce small, battery or instruction cache efficient are usually not you want on servers.

My toolchain for both is managed by the Nix derivation wrappers. It’s trivial to set custom C/C++/Rust/Go/NodeJS flags. As long as I have one flexible build tool, I can customize it for both server and embedded.

Re: Is it time for a new Embedded Linux build system?

#79

Post by KingMachiavelli »

jo-m wrote: post #54
KingMachiavelli wrote: post #17

I've been using Nix for "embedded" systems for few years now and it works fine. I don't quite understand why "embedded" has historically meant learning a completely separate tool. I've been building for x86 and ARM servers for 5+ years now, so why should targeting an ARM board be any different? Like the article points out, the software stacks people use for embedded devices are the same as people use non-embedded use…

I was thinking the same. However, reading through the docs, the author clearly knows about Nix/Nixpks and has factored some of its properties into the design: https://docs.yoebuild.org/nix.html

Actually, quite an honest and in-depth review with many genuine good points about the state of Nix. If you genuinely need sub 1GB image sizes, you would have to reimplement many of the Nixpkgs conventions like glibc/systemd.

I’d still probably just implement this in Nix/Nixpkgs so I could leverage the rest of the ecosystem but that’s not trivial at all.

I wouldn’t even say Nix as a language is better than Starlark. The author is certainly well informed in the choices made.

Re: Is it time for a new Embedded Linux build system?

#80

Post by nrdvana »

drdexebtjl wrote: post #48
nrdvana wrote: post #43

Earlier quoted context omitted.

It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI, then start a docker container and assemble a system from Alpine's package repo, then compile a kernel and a few in-house things, then extract a subset of that into an image. No cross-compile, and most of the useful software is in pre-built binary packages with the compile-time opt…

I don’t have counter examples, but hear me out. What if this hegemony in ISAs is partially because of the high costs of porting software and toolchains? Maybe the next generation of embedded devices will have ISAs that are, say, extremely optimized for real time industrial controllers, but terrible at compiling C++. It seems a little silly to give up on cross-compilation just because _today_ everything runs on ARM. E…

Well, I'd say cross-compiling is at least a little problematic... if nothing else, it is a hurdle to running automated tests. And cross-compiling still needs to exist for the people building the package repositories. I'm just sort of advocating that if you have limited amounts of time but want to put together a system image to run in an embedded or embedded-adjacent manner, the least pain / most agile solution is by building on top of a popular binary distro and then using native compilation on appropriate hardware inside a chroot or container that resembles the target.
Post reply on HN