Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Cross-Platform GitHub Action

github.com

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Show HN: Cross-Platform GitHub Action

#11

Post by saurik »

adobrawy wrote: post #8
saurik wrote: post #6

Earlier quoted context omitted.

Why do you need to build on ARM to target ARM? I support ARM and I just build on x86_64.

I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. Did you mean cross-compilations for compiled languages? It doesn't fit Python.

I am sorry but I am having a difficult time understanding your comment. If you are coding in Python, why does the underlying architecture matter at all?

Re: Show HN: Cross-Platform GitHub Action

#12

Post by saurik »

Arnavion wrote: post #10
saurik wrote: post #9

Earlier quoted context omitted.

You just need a sysroot for that distribution... you actually don't want to compile ON that distribution as it makes having a consistent and modern/working toolchain a lot more difficult. Notably, I compile for CentOS 6 (ancient, right?) on whatever the latest version of Ubuntu is--using the most recent versions of clang and rust and whatever--and simply build my sysroot using this trivial script. https://github.com/…

>You just need a sysroot for that distribution... you actually don't want to compile ON that distribution We compile in a Docker container of the distro. It's the same concept as having a sysroot. >as it makes having a consistent toolchain a lot more difficult. >using the most recent versions of clang and rust and whatever The external dependencies I mentioned are libraries like glibc or openssl, where you want to us…

> We compile in a Docker container of the distro. It's the same concept as having a sysroot.

It isn't the same concept as using a sysroot, as you are now using the compiler from that potentially-ancient different distribution. I see people constantly twisting themselves into knots being like "CentOS has some broken/limited build of gcc/clang so I can't use X" and it is like "if there is one thing you as a developer control it is the compiler".

> The external dependencies I mentioned are libraries like glibc or openssl, where you want to use the distro version.

I understood this: that's what you put in your sysroot (and is easily seen in the example script I showed for CentOS 6; my scripts for building Ubuntu sysroots are much more complicated as I am trying to avoid nesting docker and so now have some crazy fallback involving debootstrap and proot).

> Similarly the packages should be consistent with what the users of the distro would build themselves if they used rpmbuild / debbuild / whatever.

Ok, this is where we differ, then: I am not trying to simultaneously support users building on CentOS using "rpmbuild". I want to be able to use the latest build of C++ and Rust and Python and essentially have a "modern"/easy development stack but "target"/support other distributions so they can install my software as packages.

Given the goal of being able to provide source code that can itself be compiled on these random distributions I see the problem. I personally feel like it would still be better to work around that by requesting (which can itself be automated by your build) that the user compile their own toolchain that has more functionality, though.

Re: Show HN: Cross-Platform GitHub Action

#13

Post by dbingham »

saurik wrote: post #11
adobrawy wrote: post #8

Earlier quoted context omitted.

I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. Did you mean cross-compilations for compiled languages? It doesn't fit Python.

I am sorry but I am having a difficult time understanding your comment. If you are coding in Python, why does the underlying architecture matter at all?

The architecture of the docker image matters. If you build a docker image on an ARM machine (like a m1 or m2 Mac) you can't run it in x86 architecture (like a T3 or M5 AWS instance). If you build on x86 architecture, you can't run it on ARM architecture.

That is, unless you use Docker's cross platform build capability (buildx). Which was still considered experimental the last time I looked at it (about a year ago).

Re: Show HN: Cross-Platform GitHub Action

#14

Post by Arnavion »

saurik wrote: post #12
Arnavion wrote: post #10

Earlier quoted context omitted.

>You just need a sysroot for that distribution... you actually don't want to compile ON that distribution We compile in a Docker container of the distro. It's the same concept as having a sysroot. >as it makes having a consistent toolchain a lot more difficult. >using the most recent versions of clang and rust and whatever The external dependencies I mentioned are libraries like glibc or openssl, where you want to us…

> We compile in a Docker container of the distro. It's the same concept as having a sysroot. It isn't the same concept as using a sysroot, as you are now using the compiler from that potentially-ancient different distribution. I see people constantly twisting themselves into knots being like "CentOS has some broken/limited build of gcc/clang so I can't use X" and it is like "if there is one thing you as a developer c…

>it is like "if there is one thing you as a developer control it is the compiler".

>I understood this: that's what you put in your sysroot

You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster.

The correct and least-mental-overhead way of building packages for $distro is to build on $distro. The fact that users of $distro can then build your package themselves using the standard distro method is an extra benefit.

Re: Show HN: Cross-Platform GitHub Action

#15

Post by Arnavion »

dbingham wrote: post #13
saurik wrote: post #11

Earlier quoted context omitted.

I am sorry but I am having a difficult time understanding your comment. If you are coding in Python, why does the underlying architecture matter at all?

The architecture of the docker image matters. If you build a docker image on an ARM machine (like a m1 or m2 Mac) you can't run it in x86 architecture (like a T3 or M5 AWS instance). If you build on x86 architecture, you can't run it on ARM architecture. That is, unless you use Docker's cross platform build capability (buildx). Which was still considered experimental the last time I looked at it (about a year ago).

You can build other-arch images with regular `build`. You'll of course need QEMU hooked up through binfmt to be able to execute `RUN` steps while building the image, but you can do that yourself without involving `buildx`.

Re: Show HN: Cross-Platform GitHub Action

#16

Post by hk1337 »

saurik wrote: post #11
adobrawy wrote: post #8

Earlier quoted context omitted.

I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. I need to build images for an application in Python. Cross-build Docker images? I haven't heard of anything like that. Did you mean cross-compilations for compiled languages? It doesn't fit Python.

I am sorry but I am having a difficult time understanding your comment. If you are coding in Python, why does the underlying architecture matter at all?

Some packages it doesn’t matter the architecture but others it actually does. Pandas, for example.

Re: Show HN: Cross-Platform GitHub Action

#17

Post by ben-schaaf »

Arnavion wrote: post #14
saurik wrote: post #12

Earlier quoted context omitted.

> We compile in a Docker container of the distro. It's the same concept as having a sysroot. It isn't the same concept as using a sysroot, as you are now using the compiler from that potentially-ancient different distribution. I see people constantly twisting themselves into knots being like "CentOS has some broken/limited build of gcc/clang so I can't use X" and it is like "if there is one thing you as a developer c…

>it is like "if there is one thing you as a developer control it is the compiler". >I understood this: that's what you put in your sysroot You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster. The correct and least-mental-overhead way of building packages for $distro is to build on $distro. The fact that users of $di…

> You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster.

What libraries on Linux don't use the SystemV C ABI on x86?

Re: Show HN: Cross-Platform GitHub Action

#18

Post by Arnavion »

ben-schaaf wrote: post #17
Arnavion wrote: post #14

Earlier quoted context omitted.

>it is like "if there is one thing you as a developer control it is the compiler". >I understood this: that's what you put in your sysroot You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster. The correct and least-mental-overhead way of building packages for $distro is to build on $distro. The fact that users of $di…

> You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster. What libraries on Linux don't use the SystemV C ABI on x86?

What does your question have to do with this conversation?

Re: Show HN: Cross-Platform GitHub Action

#19

Post by ben-schaaf »

Arnavion wrote: post #18
ben-schaaf wrote: post #17

Earlier quoted context omitted.

> You're advocating mixing compilers and libraries that are not from the same distro release. While it may work for you it's generally a recipe for disaster. What libraries on Linux don't use the SystemV C ABI on x86?

What does your question have to do with this conversation?

Mixing compilers is not a problem because they use the same standard ABI. You often need to link against the oldest supported version of a library, or dlsym what you need, but you certainly don't need to compile per-distro. The only big exception I know of is musl.

We've been happily compiling a single executable using ~latest clang for years with very few problems beyond using a symbol that doesn't exist on older distros.

Re: Show HN: Cross-Platform GitHub Action

#20

Post by Arnavion »

ben-schaaf wrote: post #19
Arnavion wrote: post #18

Earlier quoted context omitted.

What does your question have to do with this conversation?

Mixing compilers is not a problem because they use the same standard ABI. You often need to link against the oldest supported version of a library, or dlsym what you need, but you certainly don't need to compile per-distro. The only big exception I know of is musl. We've been happily compiling a single executable using ~latest clang for years with very few problems beyond using a symbol that doesn't exist on older di…

>Mixing compilers is not a problem because they use the same standard ABI.

So first, you said:

>>What libraries on Linux don't use the SystemV C ABI on x86?

... even though the conversation was about ARM.

Second, you seem to assume that there could be no other problems other than ABI, even though...

>with very few problems beyond using a symbol that doesn't exist on older distros.

... Oh, I shouldn't even need to explain it, because you're already aware of it. So why are you acting contrarian?

When you use gcc version X then it is allowed to generate calls to libgcc version X. If the distro has libgcc version Y, then either you put libgcc version Y in the sysroot and force gcc version X to use it, which gcc version X is not prepared for, or you let gcc version X link the binary to symbols from libgcc version X, and the binary explodes when running against libgcc version Y on the actual system.

Furthermore, if a distro ships gcc version Y and some distro library version Z, it expects you to compile code that uses distro library version Z with gcc version Y. If there are compiler bugs or library bugs, the distro might have patched one or the other to ensure that they work together. By instead compiling with gcc version X (from a different distro, no less) you're breaking that guarantee and again can have silent miscompilations or other runtime issues.

I'm amazed I even have to argue about such basic knowledge. If you don't believe me, maybe you'll believe OSDev?

https://wiki.osdev.org/Libgcc

>Can I use the Linux libgcc?

>You must use the correct libgcc that came with your cross-compiler. Whatever else libgcc you found likely has a different target, was built with different machine compile options, has dependencies on the standard library, is part of a different compiler revision (your distribution may have patched its gcc, even). It is possible that using a different libgcc will work, but perhaps not reliably.

So congratulations on having YOLO'd with no problems "for years", but doing it properly is much easier for correctness and peace of mind, so excuse me if I don't follow suit.

Post reply on HN