Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving SciPy to the Meson Build System

labs.quansight.org

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Moving SciPy to the Meson Build System

#21

Post by prpl »

mumblemumble wrote: post #18
ihnorton wrote: post #17

Earlier quoted context omitted.

Nix is likely a nonstarter because as far as I can tell it does not natively support Visual Studio and MSVC. I suspect Bazel was ruled out because it requires the JVM and it has limited open source uptake relative to CMake (huge open source userbase), and Meson (limited presence in open source scientific software, but adopted by GNOME and systemd).

I can't speak for scipy, but I ruled Bazel out for a project that included Python components a few years ago. It was great for C, C++, and Java projects, but, at least at the time, there was no official Python support. I looked into third-party options and the feasibility of building it myself. Most of what I found was a few different conference talks and blog posts where the presenter was enthusiastically talking ab…

What was your artifacts and what did you choose instead?

Re: Moving SciPy to the Meson Build System

#22

Post by nrclark »

This is great. I used to support an line of Linux-based realtime audio products, and our engineers wanted me to include Scipy in the internal developer builds. It turned out to be a really tough ask, because of Scipy's build system. The result was a tech-debt blemish on an otherwise excellent codebase.

Scipy is very, very hard to cross-compile. Like, almost impossible. I eventually had to compile it on-target and store the outputs for inclusion in later images.

Distutils in general is wretched at cross-compilation tasks, should be retired.

Re: Moving SciPy to the Meson Build System

#23

Post by SAI_Peregrinus »

kortex wrote: post #8
qznc wrote: post #5

Earlier quoted context omitted.

It is Cmake with Python syntax.

That doesn't strike me as a good characterization. Especially because both Python and Cmake are quite imperative. Meson is declarative with an emphasis on immutability. It's "cmake in role only" - both are build systems, that's where the similarity falls apart. The syntax is very pythonic, however, I think that's true of many ergonomic centric formats.

CMake is two smaller systems in a trenchcoat, with a creepy stalker following along.

There's the 3.0+ CMakeLists.txt syntax, which is a nice declarative target-based system.

There's the *.cmake language syntax for finding dependencies and other scripting, which is a C'thonian horror mix of imperative and declarative bits that the trench coat tries to hide.

Then there's the creepy pre-3.0 stalker, the old syntax for both. Non-target based, imperative bits sneaking in everywhere, makes Autotools look sane.

Re: Moving SciPy to the Meson Build System

#24

Post by vips7L »

ho_schi wrote: post #3

I'm using Meson for a project using C++ with Gtkmm, Libmicrohttpd, catch2 and other libraries. My experience is good neglecting some minor issues. Meson is fast, well readable if written well, uses system libraries by default and allows flexible usage. On the other side I'm using Java with Maven and it is - a big burden. It's build in dependency retrieval system also isn't helping. Maybe I just don't like XML - becau…

I find pom.xml much more readable than deciphering a giant script file like build.gradle or CMakeLists.txt. 99% of the time I automatically know how the build works because its convention.

Re: Moving SciPy to the Meson Build System

#25

Post by misnome »

This is interesting to read. I've used CMake on-and-off for mixed python/C++ code bases, and in particular https://scikit-build.readthedocs.io/en/latest/ which aims to integrate with cmake (lightly mentioned here but not evaluated as "getting away from setuptools".

But I have never been satisfied with the solutions for out-of-tree builds with python - being able to debug-in-place extensions _and_ eventually install in a single package structure seems to be somewhat in conflict with the way that Python allows you to structure packaging, without lots of environment variable hacks.

Re: Moving SciPy to the Meson Build System

#26

Post by kvnhn »

> Cross-compiling will become possible. For years we've told people "sorry, distutils wasn't really made for cross-compiling, let us know if you have any luck". As a result we've completely ignored users on some exotic platforms, and also spent a lot of time fighting with different CI systems to do native builds.

As a user (not a developer) of SciPy, this is the big win. My "exotic platform" is embedded Linux distributions via Buildroot[1]. This opens the door to many downstream libraries becoming available as well, such as pandas and scikit-learn.

[1]: https://buildroot.org/

Re: Moving SciPy to the Meson Build System

#27

Post by chpatrick »

rkrzr wrote: post #10

> Really there were only two viable candidates: CMake and Meson. Would be great to hear why those were the only two candidates that they considered (and not e.g. Bazel or Nix).

Nix works at a higher level and usually invokes CMake, make, Meson, etc. It doesn't have its own C/C++ build system.

Re: Moving SciPy to the Meson Build System

#28

Post by [deleted] »

prpl wrote: post #21
mumblemumble wrote: post #18

Earlier quoted context omitted.

I can't speak for scipy, but I ruled Bazel out for a project that included Python components a few years ago. It was great for C, C++, and Java projects, but, at least at the time, there was no official Python support. I looked into third-party options and the feasibility of building it myself. Most of what I found was a few different conference talks and blog posts where the presenter was enthusiastically talking ab…

What was your artifacts and what did you choose instead?

[deleted]

Re: Moving SciPy to the Meson Build System

#29

Post by mumblemumble »

dfinninger wrote: post #19
mumblemumble wrote: post #18

Earlier quoted context omitted.

I can't speak for scipy, but I ruled Bazel out for a project that included Python components a few years ago. It was great for C, C++, and Java projects, but, at least at the time, there was no official Python support. I looked into third-party options and the feasibility of building it myself. Most of what I found was a few different conference talks and blog posts where the presenter was enthusiastically talking ab…

Our team started moving to Bazel two years ago with 0.21 or something. Python support is much, much better these days. We’re building Python with requirements and packing it into Docker containers. It went from “maybe run stdlib Python in an activated venv“ to actually working.

That's great to hear. Yeah, 2 years ago would have been after I made this decision. Looking forward to taking another look at Bazel; I strenuously wanted it to be the one.

Re: Moving SciPy to the Meson Build System

#30

Post by zaphirplane »

0x138d5 wrote: post #15
ho_schi wrote: post #3

I'm using Meson for a project using C++ with Gtkmm, Libmicrohttpd, catch2 and other libraries. My experience is good neglecting some minor issues. Meson is fast, well readable if written well, uses system libraries by default and allows flexible usage. On the other side I'm using Java with Maven and it is - a big burden. It's build in dependency retrieval system also isn't helping. Maybe I just don't like XML - becau…

Any particular reason you use Maven instead of Gradle?

Not op Each gradle task is artistically handcrafted written to capture the life experience and pain the author has experienced in their life

:) it devolves into a custom shell script

Post reply on HN