Single binary executable packages

by Ciprian Dorin Craciun (https://volution.ro/ciprian) on 

In support of software packages that come in the form of a single binary executable (statically linked or portable), that one can just copy anywhere in ${PATH} and execute, without needing sudo, or downloading half the distribution's packages as dependencies.

// permanent-link // Lobsters // HackerNews // index // RSS







TLDR

Just read the proposed solution section, which boils down to:

Let's drop the concept of software packages, and let's adopt the self-contained binary executable as the new unit of software distribution!

Modern software distribution

AKA "actually free with no-ads apps store" of the Linux and BSD worlds. (Although it remains to see for how long the "no-ads" statement will hold...)

One of the major differentiators between Linux and BSD's when compared to Windows (and to some extent OSX) is the package manager that fulfills a few important tasks:

I always remember with great dread the time when I was using Windows and I had to reinstall it: get the "install kits", both for software and drivers, which most of the time were more cherished than grandmas jewelry because these were unique priceless executables found only on the CD that came with the expansion card; then sit through endless sessions of next-next-next wizards; rinse and repeat each 6 months, or until it crashed more than 5 times a day...

And, if it weren't for Homebrew, OSX users would suffer from the same fate on every fresh install...

Also, I was recently made aware that there is Chocolatey for Windows, that seems to have ~9K packages available.

Sustainability of the status-quo

Unfortunately, especially as of late, say for the last 4 or 5 years, things have started to regress on the package management front... (No, it isn't the Sudoku solving capability, it's still there, and it seems mandatory for any non-trivial distribution-wide upgrade...)

It's that one, especially when working in a developer capacity, can't always find the software packages one is looking for (and not only the latest version, but any version thereof).

Granted, for most distributions -- Debian, ArchLinux, Gentoo, FreeBSD, OpenBSD, etc., and to some extent OpenSUSE, Ubuntu, Fedora, etc. -- package repositories are maintained exclusively by volunteers, who, in their spare time, and without any compensation (except complaints that they don't do enough), have to keep up with the upstream release cycle: patch, build, test, sometimes document, package, release, etc.

Unfortunately, their life isn't made easier, for once due to the explosion of a myriad of open source projects that have to be integrated in their distribution, but also due to a few factors that are more in the developer's corner:

Here are a few more words about the subject:

Numbers supporting the sustainability issue

OpenSUSE packages statistics

For example, on my OpenSUSE Tumbleweed laptop (with some additional development repositories enabled) there are:

In my view these numbers, namely the ration between packages that one can actually execute, and packages that are just supporting libraries or resources, will not scale with the explosion of open-source development...

GitHub projects statistics

If one searches for all projects tagged cli, one can find:

Based on these numbers, and say that only 10% of them are worthwhile (that is ~3K), say that each of these projects has on average 2 unique dependencies (not shared with other tools) (that is an extra ~6K), we get a total of ~10K more software packages that need volunteers to patch, test, build, and release...

I think it's obvious this is not sustainable.

Some non-solutions

In fact, and this has been going on for a few years already, the cracks are starting to show given how many projects suggest alternative install methods:

A possible "worse is better" solution

Which brings me to my point...

Given today's software landscape, lots of dependencies, lots of building issues, very quick release cycles, many distributions to cover, highly demanding users, I think there is only a single way out of this:

No, it's not Nix, or Guix, or Homebrew... they are just prolonging the illness...

Let's drop the concept of software package, and let's adopt the self-contained binary executable as the new unit of software distribution!

How would the workflow look like?

Like everything in life: tradeoffs

Cons

Are there disadvantages: yes!

Just don't tell the security people, they'll scream at you about vulnerabilities that won't be easily fixed through an upgrade.

Also, the executable size is huge, in the tens of MiB, or sometimes even in the hundreds...

Plus, what if the binary doesn't do what it says, and instead it's malicious? What if the packager replaced the tool with something else that does crypto-mining? Well, do you trust the developer (or the developer of a dependency deep in the dependency tree) not to steal from your crypto-wallet?

Here are a few more words about the disadvantages:

Pros

Are there advantages: yes, x10!

Not only can you easily install new updates (that will certainly include those vulnerabilities fixes the security people yell about), but you can now:

Limitations

And of course, this isn't suitable for everything.

I don't think a rich GUI application like LibreOffice, Firefox, Thunderbird, Slack, Discord, Skype, GMail, etc., can be delivered as a single executable.

Why, they need resources, and libraries, and HTML, and JS, and CSS, and sounds, and half the fluffy cat pictures on the internet, all to implement "features that allow users to express their most authentic selves and to bring them joy while" doing whatever users do with these wonderful life-enriching software jewels of the modern world... (The quotation is from Firefox 94 "Colorways" release note.)

Prerequisites for single binary executables

What would be the prerequisites for something like this to work?

Not a novel discovery, already in use

This is just for toy projects you say?

Here are a few examples:

Sidenote about Go's suitability for such use-cases

Hmm... Have you noticed something?

Wonder why?

A tale for another time... But until then, here's a hint though...

(Disclaimer: I kind of hate Go for various reasons... But I do use it mostly for the reason above...)

Languages suitable for single binary executables

So, say I've convinced you about this single binary executable practice, how should you proceed?

Depending on the programming language:

Concluding remarks

In the end, is it all worth it? Does it solve the software distribution problem?

Well, it's complicated...

Short term

I say that in the short term, it can certainly solve our immediate software distribution problems, by just having tools that are easy to install, and just work out-of-the-box.

Long term

However, in the long term, although I don't think it's the right solution, I certainly think it pushes us in the correct direction...

As a parallel I'll take systemd: before it was adopted, our services were initialized by poorly written and error-prone bash scripts; after it was adopted, upstream developers started paying more attention to their services life-cycles and run-environments (other service dependencies, start, stop, restart, reload, child processes, logging, etc.). At the same time, I don't think systemd (in its current form) is the right answer. On the other hand, at least it forced us to clean up this part of our software...

Now, coming back to the single binary executable, although I don't think in the long term it's the right solution, I do think it will force developers to pay more attention to their dependencies, build / test workflows, and in general improve their projects from a logistical point of view.