Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a Cargo-like build tool for C/C++

github.com

Post reply on HN
Print view View original

101–110 of 181 posts

Re: Show HN: I built a Cargo-like build tool for C/C++

#101

Post by tempest_ »

eqvinox wrote: post #90
CoastalCoder wrote: post #88

Earlier quoted context omitted.

> Never ever build in '-03 -march=native' by default. This is always a red flag and a sign of immaturity. Perhaps you can see how there are some assumptions baked into that statement.

What assumptions would that be? Shipping anything built with -march=native is a horrible idea. Even on homogeneous targets like one of the clouds, you never know if they'll e.g. switch CPU vendors. The correct thing to do is use microarch levels (e.g. x86-64-v2) or build fully generic if the target architecture doesn't have MA levels.

I build on the exact hardware I intend to deploy my software to and ship it to another machine with the same specs as the one it was built on.

I am willing to hear arguments for other approaches.

Re: Show HN: I built a Cargo-like build tool for C/C++

#102

Post by eqvinox »

tempest_ wrote: post #101
eqvinox wrote: post #90

Earlier quoted context omitted.

What assumptions would that be? Shipping anything built with -march=native is a horrible idea. Even on homogeneous targets like one of the clouds, you never know if they'll e.g. switch CPU vendors. The correct thing to do is use microarch levels (e.g. x86-64-v2) or build fully generic if the target architecture doesn't have MA levels.

I build on the exact hardware I intend to deploy my software to and ship it to another machine with the same specs as the one it was built on. I am willing to hear arguments for other approaches.

I'm willing to hear arguments for your approach?

it certainly has scale issues when you need to support larger deployments.

[P.S.: the way I understand the words, "shipping" means "passing it off to someone else, likely across org boundaries" whereas what you're doing I'd call "deploying"]

Re: Show HN: I built a Cargo-like build tool for C/C++

#103

Post by Teknoman117 »

adev_ wrote: post #49

Feedback of someone who is used to manage large (>1500) software stack in C / C++ / Fortran / Python / Rust / etc: - (1) Provide a way to compile without internet access and specify the associated dependencies path manually. This is absolutely critical. Most 'serious' multi-language package managers and integration systems are building in a sandbox without internet access for security reasons and reproducibility reas…

As someone who has also spent two decades wrangling C/C++ codebases, I wholeheartedly agree with every statement here.

I have an even stronger sentiment regarding cross compilation though - In any build system, I think the distinction between “cross” and “non-cross” compilation is an anti-pattern.

Always design build systems assuming cross compilation. It hurts nothing if it just so happens that your host and target platform/architecture end up being the same, and saves you everything down the line if you need to also build binaries for something else.

Re: Show HN: I built a Cargo-like build tool for C/C++

#104

Post by sebastos »

The tough truth is that there already is a cargo for C/C++: Conan2. I know, python, ick. I know, conanfile.py, ick. But despite its warts, Conan fundamentally CAN handle every part of the general problem. Nobody else can. Profiles to manage host vs. target configuration? Check. Sufficiently detailed modeling of ABI to allow pre-compiled binary caching, local and remote? Check, check, check. Offline vs. Online work modes? Check. Building any relevant project via any relevant build system, including Meson, without changes to the project itself? Check. Support for pulling build-side requirements? Check. Version ranges? Check. Lockfiles? Check. Closed-source, binary-only dependencies? Check.

Once you appreciate the vastness of the problem, you will see that having a vibrant ecosystem of different competing package managers sucks. This is a problem where ONE standard that can handle every situation is incalculably better than many different solutions which solve only slices of the problem. I don't care how terse craft's toml file is - if it can't cross compile, it's useless to me. So my project can never use your tool, which implies other projects will have the same problem, which implies you're not the one package manager / build system, which means you're part of the problem, not the solution. The Right Thing is to adopt one unilateral standard for all projects. If you're remotely interested in working on package managers, the best way to help the human race is to fix all of the outstanding things about Conan that prevent it from being the One Thing. It's the closest to being the One Thing, and yet there are still many hanging chads:

- its terribly written documentation

- its incomplete support for editable packages

- its only nascent support for "workspaces"

- its lack of NVIDIA recipes

If you really can't stand to work on Conan (I wouldn't blame you), another effort that could help is the common package specification format (CPS). Making that a thing would also be a huge improvement. In fact, if it succeeds, then you'd be free to compete with conan's "frontend" ergonomics without having to compete with the ecosystem.

Re: Show HN: I built a Cargo-like build tool for C/C++

#105

Post by zahllos »

tempest_ wrote: post #101
eqvinox wrote: post #90

Earlier quoted context omitted.

What assumptions would that be? Shipping anything built with -march=native is a horrible idea. Even on homogeneous targets like one of the clouds, you never know if they'll e.g. switch CPU vendors. The correct thing to do is use microarch levels (e.g. x86-64-v2) or build fully generic if the target architecture doesn't have MA levels.

I build on the exact hardware I intend to deploy my software to and ship it to another machine with the same specs as the one it was built on. I am willing to hear arguments for other approaches.

Not the OP, but: -march says the compiler can assume that the features of that particular CPU architecture family, which is broken out by generation, can be relied upon. In the worst case the compiler could in theory generate code that does not run on older CPUs of the same family or from different vendors.

-mtune says "generate code that is optimised for this architecture" but it doesn't trigger arch specific features.

Whether these are right or not depends on what you are doing. If you are building gentoo on your laptop you should absolutely -mtune=native and -march=native. That's the whole point: you get the most optimised code you can for your hardware.

If you are shipping code for a wide variety of architectures and crucially the method of shipping is binary form then you want to think more about what you might want to support. You could do either: if you're shipping standard software pick a reasonable baseline (check what your distribution uses in its cflags). If however you're shipping compute-intensive software perhaps you load a shared object per CPU family or build your engine in place for best performance. The Intel compiler quite famously optimised per family, included all the copies in the output and selected the worst one on AMD ;) (https://medium.com/codex/fixing-intel-compilers-unfair-cpu-d...)

Re: Show HN: I built a Cargo-like build tool for C/C++

#106

Post by looneysquash »

sebastos wrote: post #104

The tough truth is that there already is a cargo for C/C++: Conan2. I know, python, ick. I know, conanfile.py, ick. But despite its warts, Conan fundamentally CAN handle every part of the general problem. Nobody else can. Profiles to manage host vs. target configuration? Check. Sufficiently detailed modeling of ABI to allow pre-compiled binary caching, local and remote? Check, check, check. Offline vs. Online work mo…

> The tough truth is that there already is a cargo for C/C++: Conan2

Is it though?

When I read the tutorial: https://docs.conan.io/2/tutorial/consuming_packages/build_si...

It says to hand write a `CMakeLists.txt` file. This is before it has me create a `conanfile.txt` even.

I have the same complaint about vcpkg.

It seems like it takes: `(conan | vcpkg) + (cmake | autotools) + (ninja | make)` to do the basics what cargo does.

Re: Show HN: I built a Cargo-like build tool for C/C++

#107

Post by dijit »

tempest_ wrote: post #101
eqvinox wrote: post #90

Earlier quoted context omitted.

What assumptions would that be? Shipping anything built with -march=native is a horrible idea. Even on homogeneous targets like one of the clouds, you never know if they'll e.g. switch CPU vendors. The correct thing to do is use microarch levels (e.g. x86-64-v2) or build fully generic if the target architecture doesn't have MA levels.

I build on the exact hardware I intend to deploy my software to and ship it to another machine with the same specs as the one it was built on. I am willing to hear arguments for other approaches.

What?! seriously?!

I’ve never heard of anyone doing that.

If you use a cloud provider and use a remote development environment (VSCode remote/Jetbrains Gateway) then you’re wrong: cloud providers swap out the CPUs without telling you and can sell newer CPUs at older prices if theres less demand for the newer CPUs; you can’t rely on that.

To take an old naming convention, even an E3-Xeon CPU is not equivalent to an E5 of the same generation. I’m willing to bet it mostly works but your claim “I build on the exact hardware I ship on” is much more strict.

The majority of people I know use either laptops or workstations with Xeon workstation or Threadripper CPUs— but when deployed it will be a Xeon scalable datacenter CPU or an Epyc.

Hell, I work in gamedev and we cross compile basically everything for consoles.

Re: Show HN: I built a Cargo-like build tool for C/C++

#108

Post by ninkendo »

dijit wrote: post #107
tempest_ wrote: post #101

Earlier quoted context omitted.

I build on the exact hardware I intend to deploy my software to and ship it to another machine with the same specs as the one it was built on. I am willing to hear arguments for other approaches.

What?! seriously?! I’ve never heard of anyone doing that. If you use a cloud provider and use a remote development environment (VSCode remote/Jetbrains Gateway) then you’re wrong: cloud providers swap out the CPUs without telling you and can sell newer CPUs at older prices if theres less demand for the newer CPUs; you can’t rely on that. To take an old naming convention, even an E3-Xeon CPU is not equivalent to an E5…

… not everyone uses the cloud?

Some people, gasp, run physical hardware, that they bought.

Re: Show HN: I built a Cargo-like build tool for C/C++

#109

Post by bsder »

Teknoman117 wrote: post #103
adev_ wrote: post #49

Feedback of someone who is used to manage large (>1500) software stack in C / C++ / Fortran / Python / Rust / etc: - (1) Provide a way to compile without internet access and specify the associated dependencies path manually. This is absolutely critical. Most 'serious' multi-language package managers and integration systems are building in a sandbox without internet access for security reasons and reproducibility reas…

As someone who has also spent two decades wrangling C/C++ codebases, I wholeheartedly agree with every statement here. I have an even stronger sentiment regarding cross compilation though - In any build system, I think the distinction between “cross” and “non-cross” compilation is an anti-pattern. Always design build systems assuming cross compilation. It hurts nothing if it just so happens that your host and target…

> In any build system, I think the distinction between “cross” and “non-cross” compilation is an anti-pattern.

This is one of the huge wins of Zig. Any Zig host compiler can produce output for any supported target. Cross compiling becomes straightforward.

Re: Show HN: I built a Cargo-like build tool for C/C++

#110

Post by adev_ »

PufPufPuf wrote: post #93
CoastalCoder wrote: post #88

Earlier quoted context omitted.

> Never ever build in '-03 -march=native' by default. This is always a red flag and a sign of immaturity. Perhaps you can see how there are some assumptions baked into that statement.

The only time I used -march=native was for a university assignment which was built and evaluated on the same server, and it allowed juicing an extra bit of performance. Using it basically means locking the program to the current CPU only. However I'm not sure about -O3. I know it can make the binary larger, not sure about other downsides.

> The only time I used -march=native

It is completely fine to use -march=native, just do not make it the default for someone building your project.

That should always be something to opt-in.

The main reason is that software are a composite of (many) components. It becomes quickly a pain in the ass of maintainability if any tiny library somewhere try to sneak in '-march=native' that will make the final binary randomly crash with an illegal instruction error if executed on any CPU that is not exactly the same than the host.

When you design a build system configuration, think for the others first (the users of your software), and yourself after.

Post reply on HN