Building binary deb packages: a practical guide • Hacker News Board

Re: Building binary deb packages: a practical guide

#31

Post by Conan_Kudo » Wed, Jun 03, 2020, 4:10 AM UTC

infinity0 wrote: post #4

Athas wrote: post #2

I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Debian packages. I've been a satisfied Debian user for over ten years, but I gave up on trying to package my own software. I could certainly figure out how to manually create a `.deb` with an executable and some man pages, but not how to automate the creation of…

As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assura…

> The tradeoff you get with other distributions being quick, is less quality assurance

The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right.

Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE.

One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base.

As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform.

I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...).

[1]: https://github.com/debbuild/debbuild

[2]: https://github.com/debbuild/debbuild-macros

(Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)

Re: Building binary deb packages: a practical guide

#32

Post by pabs3 » Wed, Jun 03, 2020, 5:05 AM UTC

Conan_Kudo wrote: post #31

infinity0 wrote: post #4

Earlier quoted context omitted.

As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assura…

> The tradeoff you get with other distributions being quick, is less quality assurance The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right. Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more…

Getting debbuild into Debian would be relatively simple, but getting folks to switch existing packages away from debhelper is unlikely to be easy.

https://mentors.debian.net/intro-maintainers

Re: Building binary deb packages: a practical guide

#33

Post by pabs3 » Wed, Jun 03, 2020, 5:14 AM UTC

gorgoiler wrote: post #22

j1elo wrote: post #21

Debian packaging lacks a practical guide that is comprehensive enough to cover all modern use cases, but at the same time doesn't lack focus and goes to the point. The official docs are very extensive, but then they lack self-references to guide the reader to the interesting parts, or surprisingly lacks information that one would expect to find in there. You will read about debian/rules file, but then open one from a…

Where do Debian maintainers collect their institutional knowledge, these days? In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project? Nothing beats being able to read the code for learning how a system actually works.

Some packages use git for maintaining the packaging, but the Debian archive and build servers are still based on Debian source packages (.dsc files).

Re: Building binary deb packages: a practical guide

#34

Post by pabs3 » Wed, Jun 03, 2020, 5:15 AM UTC

gorgoiler wrote: post #22

j1elo wrote: post #21

Debian packaging lacks a practical guide that is comprehensive enough to cover all modern use cases, but at the same time doesn't lack focus and goes to the point. The official docs are very extensive, but then they lack self-references to guide the reader to the interesting parts, or surprisingly lacks information that one would expect to find in there. You will read about debian/rules file, but then open one from a…

Where do Debian maintainers collect their institutional knowledge, these days? In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project? Nothing beats being able to read the code for learning how a system actually works.

Debian maintainers collect their institutional knowledge from participating in package maintenance, following Debian mailing lists and other communication forums etc.

Re: Building binary deb packages: a practical guide

#35

Post by pabs3 » Wed, Jun 03, 2020, 5:29 AM UTC

dijit wrote: post #24

infinity0 wrote: post #3

It should be noted that none of this article applies to actual development that goes on in the Debian distribution from https://www.debian.org Official Debian development, being FOSS oriented, proceeds from Debian source packages instead, and one never creates a binary package from scratch as described in the article. Instead, we use `dpkg-buildpackage` and other similar tools. The creation of the source package is t…

Is there a guide on doing things that way? I've read some of the guides for making .deb packages (as I had to do so recently) and I have to be honest, what I encountered was "messy", articles and documentation that would work if followed but they didn't look even slightly similar. I'm too embarrassed to even link to the sources that I created.

The official entry point for getting packages into Debian is here:

https://mentors.debian.net/intro-maintainers

Debian packaging documentation is unfortunately a swamp, since people continue to write new guides instead of improving existing ones.

Re: Building binary deb packages: a practical guide

#36

Post by pabs3 » Wed, Jun 03, 2020, 5:31 AM UTC

zwetan wrote: post #25

infinity0 wrote: post #3

It should be noted that none of this article applies to actual development that goes on in the Debian distribution from https://www.debian.org Official Debian development, being FOSS oriented, proceeds from Debian source packages instead, and one never creates a binary package from scratch as described in the article. Instead, we use `dpkg-buildpackage` and other similar tools. The creation of the source package is t…

what if you need to distribute a .deb containing binaries that target other operating system ? example: I got a tool that allow from any Windows/macOS/Linux to package a runtime+script to any Windows/macOS/Linux the Windows .exe can not be compiled from sources under Linux, so this particular runtime can never be distributed on Debian official?

Debian contains cross-compilers for Windows (but not macOS yet), so you can compile a .exe and build that into a .deb easily.

https://packages.debian.org/search?keywords=mingw

Probably the right thing for you to do is switch to WinGet on Windows and the Apple Store for macOS.

Re: Building binary deb packages: a practical guide

#37

Post by pabs3 » Wed, Jun 03, 2020, 5:55 AM UTC

infinity0 wrote: post #4

Athas wrote: post #2

I think this guide glosses over the most difficult aspect: writing a package whose executables are generated reasonably reproducibly, using tools from other Debian packages. I've been a satisfied Debian user for over ten years, but I gave up on trying to package my own software. I could certainly figure out how to manually create a `.deb` with an executable and some man pages, but not how to automate the creation of…

As I mentioned in the other comment, the creation of a Debian source package has a lot of barriers, some of them good and for quality purposes, some of them bad and due to historical cruft. Documentation is also not very newbie-friendly. The actual automatic reproduction of a binary from a source package is simple - `dpkg-buildpackage`. The tradeoff you get with other distributions being quick, is less quality assura…

Got any examples of barriers? Debian packaging is trending towards entirely automatic, but we aren't quite there yet and there are a lot of things we could do to change that, mostly in dpkg-dev and debhelper.

https://wiki.debian.org/AutomaticPackagingTools

Re: Building binary deb packages: a practical guide

#38

Post by mavu » Wed, Jun 03, 2020, 7:41 AM UTC

This is better: https://github.com/phusion/debian-packaging-for-the-modern-d...

(someone with the applicable knowledge please fork and continue it)

Re: Building binary deb packages: a practical guide

#39

Post by debiandev » Wed, Jun 03, 2020, 8:18 AM UTC

gorgoiler wrote: post #22

j1elo wrote: post #21

Debian packaging lacks a practical guide that is comprehensive enough to cover all modern use cases, but at the same time doesn't lack focus and goes to the point. The official docs are very extensive, but then they lack self-references to guide the reader to the interesting parts, or surprisingly lacks information that one would expect to find in there. You will read about debian/rules file, but then open one from a…

Where do Debian maintainers collect their institutional knowledge, these days? In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project? Nothing beats being able to read the code for learning how a system actually works.

DD here.

apt-get install maint-guide ...and then read:

file:///usr/share/developers-reference/index.html file:///usr/share/doc/maint-guide/html/index.en.html

Also look at existing packages and talk to DDs. Avoid random wikis.

I worked on many deployment systems and reading these guides made me a better engineer overall.

Re: Building binary deb packages: a practical guide

#40

Post by sm_ts » Wed, Jun 03, 2020, 8:22 AM UTC

Athas wrote: post #11

pizza234 wrote: post #10

Earlier quoted context omitted.

This post describes how to do that, via PPAs: https://is.gd/Eu55IG .

Thanks. That may have been useful to me once, although my motivation was to get my software in the main Debian package set, which I think is quite a different thing from PPAs. Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: htt…

> Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools

I agree that Debian packaging is difficult/odd/arguably obtuse; I guess it's a legacy matter. Your phrasing can be interpreted as if the guide itself is obtuse, it's not clear :)

> including apparently modifying some file in /etc and running other stuff as root?

It's important to read and understand it fully. Anything you do it as root is required only to run a preview build locally. This is an optional step, so no, one doesn't need to perform admin configurations/operations if they don't want to, although performing a local build is a desirable step.

> In contrast, this was my first contribution to Nixpkgs

This is not a proper comparison, as you're comparing an entirely manual procedure versus using a packaging tool.

A proper (closer) comparison is probably against the tool that the guide references: https://github.com/saveriomiroddi/ppa_packaging, where you just declare all the metadata, and invoke a script that takes care of everything.

All in all, I'm personally puzzled about why there aren't tools around to simplify/automate Debian packaging. I've started the small tool mentioned above (`ppa_packaging`), but I'll likely not continue working on it, as my experience with niche projects is that, while useful in their (narrow) domain, they're very resource-intensive to maintain.