Perfecting Rust Packaging

[brson]

@gus

So: my "source-based Rust libraries" plan is to have:

Thanks for the clarifications! I think I have a decent grasp on it now.

[brson]

@gus

Default CPU features. @gus pointed out recently that rustc's default code generation is for i686 while Debian's 32-bit x86 distro targets i586-class machines. Should we be adjusting the target specs for these systems? How? Any insight into how other compilers deal with this?

In general, I think we're going to continue to define new target triples for various additional platforms and variants. Some sort of wiki/doc for tracking them would be good, but perhaps just using comments in the existing mk/cfg/* files is the right place.

@gus If we provided an out-of-the-box i586-unknown-linux-gnu triple that Debian could configure with, and didn't use i686 features, would that work?

[brson]

Thanks for everybody’s patience. I’ve posted a new thread with a bunch of tasks taken from this thread, and naturally I’d appreciate review.

[gus]

@brson

@gus If we provided an out-of-the-box i586-unknown-linux-gnu triple that Debian could configure with, and didn't use i686 features, would that work?

It would certainly save me the effort!

[ahmedcharles]

@gus

• The Debian "binary" package for a Rust library will just be the source, installed in a known directory somewhere (ie: the Debian package build step is basically a no-op).

I would have thought that the -dev package would contain the source and the binary package would be effectively empty, while having the application package depend on the empty library package. This would allow for -sys packages to have native dependencies which are dynamically linked, like openssl-sys depending on openssl and therefore, applications like cargo would depend on openssl-sys, which would be empty but would ensure that libopenssl is installed?

[mzabaluev]

@ahmedcharles

I would have thought that the -dev package would contain the source and the binary package would be effectively empty, while having the application package depend on the empty library package. This would allow for -sys packages to have native dependencies which are dynamically linked,

I don't think there is a need for empty non-dev packages. When the application is built, it gets the dynamic linkage parameters from the -sys crates which are built from -dev packages, and then the package build tools turn the dynamic library dependencies which are baked into the application binary into package dependencies automatically. You may even shed some unnecessary dependencies this way, as IIRC rustc passes option -as-needed to the linker.

[gus]

@ahmedcharles

I would have thought that the -dev package would contain the source and the binary package would be effectively empty

My nomenclature might be a bit confusing. I've been somewhat carefully trying to use these terms without ever actually defining them anywhere:

• "source package" - the input to the (Debian) packaging build process. In Debian this is a *.dsc and some tarballs, in Redhat (I think) this is a srpm. No-one other than the distro packager(s) ever see these.
• "binary package" - the output from the (Debian) packaging build process. This is a *.deb or *.rpm. This is what you apt-get/yum/whatever install.
• "library package" - a binary package that contains a library in some way. I see above I haven't used better terms than "-dev" and "not -dev" to describe the way regular C dynamically linked libraries are typically split into multiple binary packages
• "application package" - a binary package that contains an end-user (Rust) application. It most likely depends on several "not -dev" library packages.

So following the plan described earlier, a Rust library would be distributed in a "binary package" that bundles the Rust source. Since it is only a build-time dependency, it would be called "foo-dev" (or "foo-devel") and there would be no "foo" (not -dev) unless there are some additional data files, etc required at run-time to use the library.

To continue your example, a hypothetical librust-openssl-sys-dev.deb package would include the Rust source and depend on (non-Rust) libopenssl-dev.deb for the unversioned C *.so used at link time. (Non-Rust) libopenssl-dev.deb depends on libopenssl.deb and perhaps other libraries, but that isn't really any of our business. There would be no librust-openssl-sys.deb package. A hypothetical cargo.dsc source package might build-depend on librust-openssl-sys-dev.deb, which would also pull in the (non-Rust) libopenssl-dev.deb. Building the package would produce cargo.deb, which would have a run-time dependency on (non-Rust) libopenssl.deb to pick up the real openssl shared library and no run-time dependency on (Rust) openssl-sys at all because it has been statically linked. As @mzabaluev points out, the packaging build tools can usually deduce the run-time dependencies automatically (basically by running ldd over everything and seeing what dynamic libraries were actually linked).

[ahmedcharles]

@gus

As @mzabaluev points out, the packaging build tools can usually deduce the run-time dependencies automatically (basically by running ldd over everything and seeing what dynamic libraries were actually linked).

Thanks for the explanation, I didn't know this was how it worked before. That's cool.

[briansmith]

Hi, regardless of whether Linux distros package rustc and cargo or not, I expect there will be a strong need to be able to easily install newer versionn of rustc and cargo without relying on the distro. I hope, in particular, that the Rust project will distribute, at least, properly-signed and maintained RPM and DEB packages of new versionn of rust and cargo from its own repository, that can be installed on a variety of Linux distros. See https://launchpad.net/~terry.guo/+archive/ubuntu/gcc-arm-embedded for an example of this which has worked great for me and others.

Debian Stable and Red Hat are notorious for maintaining old versions of packages way longer than anybody wants to support. With the great amount of improvement to rustc and Cargo, I think we’re actually 1 or 2 years away from being able to expect any Rust library or application author to support any version of Rust older than the latest stable release. I personally don’t want to be bothered by Linux distros asking me for free help in backporting changes or adding compatibility hacks to support old versions of rustc and Cargo that they ship. I imagine other people will feel similar. Having an official PPA operated by rust-lang.org would help ease this burden so that we can spend more time focusing on creating the future instead of maintaining the past.

[brson]

Here’s a heads-up on a PR that affects downstream packaging: https://github.com/rust-lang/rust/pull/30353

This PR turns on rpath for the compiler by default in order to make the official binaries ‘just work’ without setting LD_LIBRARY_PATH. Distributions that do not want that behavior would now need to pass --disable-rpath.

[hsivonen]

@briansmith

regardless of whether Linux distros package rustc and cargo or not, I expect there will be a strong need to be able to easily install newer versionn of rustc and cargo without relying on the distro.

If distros 1) ship Firefox every six weeks (Ubuntu and Fedora do and Debian intends to AFAICT) and 2) have a policy that packages they ship have to be buildable with tools available from the distro repos, then those distros need to ship rustc often (ideally every six weeks) or there will be trouble first when Rust code in Firefox makes it to the release version and later when Rust code depending on a newer rustc makes it to subsequent release versions of Firefox.

I posted to dev-platform about this today.

What's the current status of being able to build rustc stable release and std stable release with something more predictable than a particular nightly along the way?

[aturon]

@hsivonen

What's the current status of being able to build rustc stable release and std stable release with something more predictable than a particular nightly along the way?

Some of us replied on the linked thread, but for reference: there are no known technical obstacles to adjusting the snapshot process to use the previous stable release, and the core team will formulate a plan at our next meeting (on Wednesday).

[krig]

As a +1 for this before wednesday; If each release is buildable from the previous release, that would at least be enough for us to be able to submit rustc for inclusion in openSUSE Tumbleweed (and then potentially in the next stable releases of both openSUSE and SLE). There’s more work needed to get cargo and crates packaged, but having the compiler itself available would be great.