(I suspect this discussion is going to stretch the forum tool)
Just to be clear here, I suggest the "rustc" package should also include the
Debian breaks out the rust-std libraries into a "runtime" package (runtime dylibs only) and a "-dev" package (link-time dylibs and rlibs). This is done mostly for cross-compilation technicalities and should be invisible to the user (who just apt-get installs rustc). Specifically, when cross-compiling, you need the -dev package for the host/target arch, and the runtime package for the build arch (because rustc itself needs it).
Note that we replace the link-time dylibs with symlinks to the run-time dylibs in the packaging. They aren't byte-for-byte identical because they come from different stages of the rust build process, but they're meant to be equivalent. This saves 70-ish MB on disk, from memory. I think it would be reasonable to make the upstream rustc install process always do this too.
Also note that distros typically include "source" packages, and we should probably be consistent here too (where possible). I suggest in particular using "rustc" and "cargo" as the source package names (and not "rust" and "cargo", since upstream seems to have also moved to "rustc" for the rustc source archive).
Yes, we move the files all around to fit Debian expectations. Realistically I don't think upstream can predict where that is going to be (particularly across non-Linux platforms) and you may as well just pick a standard GNU-ish layout (like you currently do).
Note Debian also has a policy of removing "web bugs" from HTML documentation for privacy reasons. Eg: we post-process the Rust docs to point to local copies of the Rust logo, etc. Again, just FYI and I don't expect upstream to do any more to accommodate this use case.
This is highly relevant to an an active Debian bug currently under discussion. Should distros package a "nightly" rustc (or some version built to allow access to "unstable" features)? I definitely think this needs some careful discussion and we should have a common policy on how/where/if we expose nightly packages.
This seems attractive at first glance, but consider that cargo, etc need to be taught to use an appropriate versioned executable before it is useful - so I suggest we don't over-engineer this without a clear need/expectation that this will be useful.
It's straightforward enough to do one or the other approach - and an earlier version of the Debian packaging had versioned executables like this. I removed it during a big package re-org because I was trying to simplify things and didn't have a clear use-case in front of me. Happy to follow the community's lead here.
Yes, all these things! Cargo releases should be clearly tagged and build with the most recent stable rust release. I don't care whether the pre-built binaries that ship in the combined upstream binary blob are built from each other, since I won't be trying to rebuild exactly them. I would prefer formal source tarball releases to just git tags (but can live with git tags).
Embedding the upstream stage0 works, and we have argued successfully to have this allowed into the Debian archive. Our current approach can't scale to all the Debian architectures, however, and is the primary reason we only support amd64+i386 architectures currently.
What I'd like to do is ship the stage0 blob for just one architecture, and use that to cross-compile all the other architectures. Complications are that you can't "just" rebuild any rustc release with itself (eg: for bootstrapping other archs by cross-compiling), although I believe this is possible by jumping into the build after stage1 (just haven't tried yet). I'd also really like to avoid re-doing this for every subsequent rustc release, which requires building subsequent rustc releases with something already in the Debian archive (ie: the immediately prior stable rustc).
I haven't thought this through much, but a hypothetical alternative to
cargo-boostrap.py might be a cargo subcommand that dumps out a list of source URLs and a shell script of rustc commands that can be run somewhere else without cargo.
I don't think we need to put much work into inter-operating with the native package manager (or phrased differently, if we try that we're going to get it wrong and it will end up awkward and a second-class-citizen anyway).
From a distro point-of-view, to ship a Rust-built executable (let's call it "mozilla") we want:
- Has to build from things (rustc, cargo) already in the archive.
- Want to have the dependencies exposed in the regular packaging metadata in some way.
Both of these are for potential security releases. If there's a security fix required to (eg)
libpng-rust, then we want to be able to easily find all packages that include
libpng-rust and recompile them.
I think that provided there's a way to tell cargo "build this source using these other sources" and have all that provided locally on disk somehow, then I think we're good. It doesn't really matter how easy it is to put those sources there - provided there is some automated way to do it.
I think it would be ok to keep the package-managed and user-cargo-managed software separate.
Huh. Does this mean the standard library will build without using unstable compiler features?
The Debian packages are already split out like this (for this exact reason). The packages are called (effectively)
libstd-rust-7d23ff90:amd64 and similar. I don't think we need any further help from upstream for this, which is why I'm a bit surprised/unsure about what changes you're suggesting.
We don't currently have any non-Debian target architectures packaged, but I plan to introduce an architecture independent (in the Debian sense)
libstd-rust-xxx-$triple for any standard Rust architectures where we also have a cross-compiler packaged. At the moment I think this is only win32/mingw, but I expect to see an Android cross compiler soon too.
In particular, I was surprised to see Rust recently jump to an unreleased version of LLVM. I thought those days were behind us - and the commit in question didn't even seem to have a discussion of why it was necessary and what breakage we should look out for (I've since learned that it was for the native-win32 target, which is probably why we didn't notice any downside).
Debian has built the last few releases against the system LLVM. rustc-1.2 required a few test patches to compile against LLVM 3.6, and we've since moved to LLVM 3.7 now that it has been released. The Rust test coverage is pretty good and so far we've relied on that to detect LLVM breakage - I'd hope that this continues to be a sufficiently thorough test.
Testing against an LLVM release would be good - but only if we also act on it when it breaks. If we're doing that then I'd question the value in vendoring LLVM at all...
In general, I think we're going to continue to define new target triples for various additional platforms and variants. Some sort of wiki/doc for tracking them would be good, but perhaps just using comments in the existing
mk/cfg/* files is the right place.
In a sense, the upstream library author "controls" the API but the cargo user (or distro builder) is the one who should define the ABI. Basically all the codegen options (
rustc -C help) might need to be tweaked by the person building the software to fit local conditions/policies/architectures.
A concrete example is the Debian hardening options. Most of these are designed for C and don't really apply to Rust, but some do - and we currently patch the rustc build process to allow us to pass
-Wl,-z,relro to the link step, for example.
In particular, cargo currently makes tweaking compiler options hard, in an (overly strong imo) attempt to ensure reproducible builds.
rust-installer is just an implementation detail of "make install", as far as I'm concerned. In particular, I move some files around to different places, and don't include the "metadata" bits that rust-installer creates.
Debian has standard tools to notice new upstream releases, so continuing to do exactly what you're doing is good. Improving the bootstrap story directly improves this, for non-amd64 architectures.
It would be slightly easier if Cargo had regular upstream tarball releases published as github releases (or on any other webpage) since the tools for polling git repos are not quite as mature/featureful.
I hold a biased position here so this is awkward to answer From my point of view, I'd hate that because then I have to also work around whatever you do in your package, and I'd rather you just contributed whatever efforts to Debian proper instead. I don't think I can fairly weigh the benefit of such a thing to an end user.