How to distro-package a (Cargo-using) Rust executable?


#1

Let me start off by saying “Yes, I know there’s no ABI stability yet”. This isn’t that conversation.

So let’s say I want to package a Rust executable for $distro (my non hypothetical example is packaging cargo itself for Debian, but the approach should be generic). How should that be done? What features are missing from cargo to make this possible/easy?

I figure the overall requirements might look like this:

  1. Must not use the network (some of the build-bots don’t have reliable network access, reliability, control, etc)

  2. Should not require any changes to Cargo.toml in executable or libraries (I think there’s already the right metadata there, and we want the end result to not require additional effort from upstream Rust developers)

  3. Rust library dependencies must be packaged/provided as source for now because ABI instability. In the future presumably they can be shipped as rlibs/dylibs but Not Yet. (golang already ships libraries as source in Debian fwiw)

  4. Distro packagers should be able to provide additional rust compile flags to meet various distro policies (See https://wiki.debian.org/Hardening for some mostly C-oriented examples, but the link options and general motivation still applies to Rust). In Cargo terms, I think this means providing rust-flags, linker, a profile, etc in a way that overrides the projects own Cargo.toml. It would be ok if we have to patch Cargo.toml as part of packaging, but nicer to be able to pass it in these build options from outside.

  5. The individual libraries should be in separate packages and visible in the distro metadata (ie: not as a massive tarball of combined sources) so bug/security fixes are trackable, and are re-usable between dependant executable packages

  6. It would be nice if multiple versions of library sources were available simultaneously to make (5) easier (is this required for anything?)

I’m assuming some experience with distro packaging above, but I’m honestly not sure what isn’t already widely known. I’ll happily expand/explain any of the motivation for the above where required.


Now I’m still learning about the more obscure features of cargo, and what’s easy given the existing codebase. I think a reasonable way forward looks like the following, but the purpose of this post is to flush out more learned advice on the topic.

Strawman proposal:

  • Each “library” package installs its source code and Cargo.toml pretty much unmodified into /usr/share/rustsrc/$name-$version (or somewhere similar).
  • The “executable” package build-depends on these in the usual distro-specific way.
  • The executable package is built using (hypothetical args) cargo --release --no-network --repo-search-path=/usr/share/rustsrc
    • Note the “repo search path” needs to find /usr/share/rustsrc/*, and specifying a list of paths to individual packages (eg: the current .cargo/config paths semantics will quickly become unwieldy)
  • Cargo finds all the packages it was looking for below the “repo search path” and treats them very much like it had just downloaded them itself. The precise details here could use some fleshing out from someone more knowledgeable.
  • CARGO_HOME is set to something “safe”, or perhaps a standard config that sets the repo search-path and other packaging-policy options, rather than using command line flags.
  • Following the CFLAGS, LDFLAGS, etc precedent, a RUSTFLAGS env var is added to rustc command lines, allowing RUSTFLAGS='-C link-args=...' and the like.
  • The distro package now just installs the resulting executable(s) in the right place in /usr/bin (or wherever). Note the resulting binary won’t have any run-time dependence on the Rust libraries because static linking is preferred, but may run-time-depend on other © libraries. The distro package should handle this just like it would for any regular (C-compiled) executable at this point (for Debian, this means dpkg-shlibdeps and friends). This step is quite uninteresting.

Does something like that seem reasonable? Discuss and then I (or someone else) will go file a bunch of bugs for the specific work items. Most of the bits I’m unclear of are around the “repo search path” mechanism, and what is easiest to fit in with the existing similar features already in Cargo.

(Thanks for reading)

  • Gus

Perfecting Rust Packaging
Perfecting Rust Packaging
Perfecting Rust Packaging - The Plan
Perfecting Rust Packaging
Perfecting Rust Packaging
#2

If I may add a request for this: I’d like programs and library to be able to ship with data files (images, stylesheets, whatever) that they’d read at run time. This requires a convention of where in the filesystem to find these files, both during development and once packaged and installed.


#3

cc @brson @alexcrichton


#4

I like the plan.

I hope we can get some momentum on packaging-related issues. Rust is a bit painful to package at the moment, and getting it into the major Linux distros around the time of 1.0 release would be awesome. Much better than the distros hacking it up separately, or waiting until after 1.0 to hash it out.

See also: stage0 in a post-rust-1.0 world


#5

Cargo #1863 is somehow related to this discussion and its implementation details.