The sorry, sorry state of Linux packaging

Some Linux things have changed a lot since the 90s…

The sorry, sorry state of Linux packaging /img/linux-package-managers.jpg

A couple of weeks ago, Steven J. Vaughan-Nichols reported what happens when Open Source developers go bad: “JavaScript developer Marak Squires wasn’t happy about not making money from his open-source libraries, so he deliberately corrupted them, leaving programmers and end-users with dead-in-the-water programs."

In an email discussion about that story Christine Hall, editor of FossForce, observed that such possibilities are one of the reasons why she advocates that “everyday desktop Linux users source their apps from their distro’s repositories whenever possible. Home users don’t have the resources and often the expertise to check out the legitimacy and efficacy of all the software they need to run on their computers, but by using the software from your distro’s repository you have a degree of assurance the software has been vetted."

For me, instead, that story was the stimulus to finally share something that has been seriously bugging me for years now.

Once upon a time…

Once upon a time (for me, the mid-1990s) if we Linux users wanted to install some extra software, we had to manually find, download, configure and compile its source code. This process, that on those slow computers with those even slower Internet connections could take several hours, was often made much slower by the dreaded “dependency hell”: the compilation process aborted because it depended on some third party library that you had to manually find, download, configure and compile first… while praying that a) it did not depend on two or three other libraries to manually find, download etc… and b) that none of those pieces was incompatible with the libraries you had already installed.

Software repositories for Linux distros: a great idea, until it WAS used

Then, “package managers” for .deb, .rpm and other software distribution formats came to the rescue. You just said “I want to run this software” and the package manager would find, download and install for you some surely compatible version of that program and all the dependencies it had. All by itself. This did not happen always, but often enough to feel like heaven, compared to the past.

Then, everything went uphill from there. In the wrong way

Those happy times ended soon. After some years we fell head first in swamp that shows no intention to dry up. Today, if you want to do anything beyond basic SOHO usage of Linux, you are very likely to end up with some nightmarish mix of Perl modules from CPAN, Python modules from pip, Ruby modules from RubyGems, JavaScript code via npm, Flatpak, Snap or Appimage binaries, random files (e.g CMS themes) to fetch with git, containers made for different versions of Docker (and I am surely forgetting something)…

because none of that stuff is packaged as .deb or .rpm anymore (*), that is with coherently declared, centrally verified dependencies and installation procedures that ONE package manager could resolve in ONE, coherent way.

From Dependency Hell to Deployment Hell

On one hand yeah, sure: today there are many more programs “packaged” for Linux than twenty years ago. On the other, I feel myself longing every year more for the good old days when lots of developers limited themselves to, you know, “we don’t release Linux binary packages of our software, because Linux distros are too fragmented and we don’t know for WHICH one we should build packages."

Today, if you need ten programs not present in the default repositories of your Linux distribution, you may likely need to run almost as many separate software distribution systems, all deliberately created to simplify your life of course, all blissfully unaware of each other.

Forget compiling from sources: it would just move the problem to installing the same number of separate, possibly uncompatible toolchains, most of which are much more complex to set up and use that the good old “./configure make && make install” of yore.

Oh, and of course you should be prepared to re-run all those systems every time you upgrade your Linux distribution. Not manually, of course! Without doubts, the Right Thing To Do (TM) would be to handle everything with some custom-made Ansible playbooks, or some other CM system, right? To add, that is, another level of embarrassing complexity to a problem that should not exist in the first place.

The sorry, sorry state of Linux packaging /img/rube-goldberg-cartoon-og.jpg

All this mess, for a HANDFUL of computers???

Two more things: first, this is not just me, or a novice-only problem. In the same discussion mentioned above Christine Hall, who has more Linux esperience than me, also shared that:

“Just the other day I had the problem of needing to remove an app that I thought had been installed as either a Snap or a Flatpak, but I couldn’t find the app using snap and flatpak commands. I knew I didn’t install it using apt, because it’s not in any of the apt-based repositories I have installed."

“It turned out it was installed as an AppImage. So yes, we’re in a mess when it comes to software sources right now."

Second, in case you missed it: please note how none of these complaints come from Fortune 500 CIOs, or system administrators of corporate networks with thousands of hosts. This is from freelancers who need to run just a handful of Linux computers.

My little Linux funding dream

Should I ever become one of those billionaires with way, way more money than they could possibly need, I would seriously consider paying a substantial annuity to each developer of every Free Software I use, on the only conditions that they:

  1. always, promptly release carefully tested .rpm and .deb packages (*) of every version of their software, for the official repositories of the main Linux distributions
  2. never, ever, ever release the same software in any of the other ways mentioned above

(*) you prefer AUR, or other similar formats? No problem!

The goal is not to have just one or two packaging formats. It is to stop being forced to use ten, mutually incompatible formats, no matter which Linux distribution one runs, which is exactly what we have today:

The sorry, sorry state of Linux packaging /img/xkcd-standards.png

Linux package management systems are the new cellphone chargers, or the new “smartphone” batteries. There HAS to be something better than this.