there's a lot of arguments for and against this, but i think a lot of these don't matter, because many crates are already shipping prebuilt artifacts.
ideally this could also be integrated with some sort of deterministic build system.
the simplest implementation would be a per-crate (not per package) table that maps a target triple and a rust version (may be ignored for crates with a stable ABI or ones that are cli-only) to a hash and url for that crate's output.
may need another layer of nesting in the table for crates that can be built as static or dynamic.
there would be options in config.toml to enable/disable prebuilt binaries, either globally or via allowlist/denylist.
for programs with no build scripts, it should be fairly easy to verify the executible via building and comparing the hash.
A big problem with pre-compiled packages is that they lock in
profile settings
RUSTFLAGS
dependency versions
This would break the current model of Cargo where the top-level package has control.
One option is to instead focus on cross-workspace caching which we could then extend to remote caching.
Another option is for us to explore an alternative style of dependency. I can't think of a specific issue tracking this but I've mentioned the idea in several places. The core of the concept is that Cargo dependencies today are akin to headers-only libraries in C++ or pure Python packages. Both C++ build systems and Python have the concept of heavier-weight packages where everything underneath that package is opaque to you, In a way this is how std works and build-std takes it further by us shipping and a Cargo.lock for std and building it. We could explore a way to generalize what build-std does to allow others to have these heavy weight dependencies. Likely the first step in this effort is build-std itself while keeping this future possibility in mind.
It should be possible, but there are many things that need to be solved first.
Cargo would need to have a deeper understanding of the relationship between the build environment and the ABI. With some cooperation from rustc, it may be done even without an officially stable ABI, under assumption that same compiler with the same ABI-relevant settings is compatible. Perhaps rustc could export machine-readable metadata about its flags, and in that info include metadata on which flags affect the ABI compatibility?
-sys crates and their downstream users would have to be excluded from pre-built binaries, since they're likely to depend on system-specific system-wide state that Rust/Cargo doesn't manage. Currently -sys crate isn't an explicit pattern, and build.rs is used for much more than sys crates, so Cargo would first need to implement some build isolation and/or declarative alternatives for common build.rs tasks.
There are quite a few different Rust/Cargo versions in use. Multiply this by OS Ć CPU architectures, and there will be lots of variations to build. Combined with enormous combinations of dependent crate versions, it could result in a huge number of libraries to build and/or a pretty poor cache hit ratio.
There are hundreds of releases of libc, serde, and other usual suspects found in lockfiles. Cargo + crates-io would need to learn how to negotiate a useful common set of dependencies that has pre-built binaries (perhaps the registry index would have extra info listing available builds?)
I'm also concerned about sizes of pre-built artifacts. The target dir of even a small project quickly grows into multiple gigabytes of disk space. Building is slow, but downloading gigabytes of rlibs for every kilobyte of source code could be even slower. For example, debug information of zero-cost abstractions is not zero cost. Rust's MIR and debug info still has traces of all of the calls that have been inlined, all of the arguments that were turned into local variables. Even though the debug format is very efficient, it still adds up to absurd amounts of data.
I think so far the assumption was that disk space is relatively cheap, and accuracy of data and simplicity of the implementations were a priority, but for downloadable artifacts it may be necessary to create a compact representation of rlibs.
Well, I'm quite serious when I say that I don't think crates should ever need to ship binary artifacts, and furthermore that the social contract of crates.io is that it serves you source and only source, with the possible, grudging exception of prebuilt core Rust toolchains, which you get via a different mechanism than anything else (rustup instead of cargo). If I caught a crate in my software's dependency graph shipping a binary artifact I would drop that crate and everything else written by the same people from my dependencies and never use them again, that's how strongly I feel about this.
You say that "many" crates are doing this. Please give us your list, and, if you can, explain why the authors of each crate on the list think they need to do this.
There's the watt crate that enable people to ship precompiled macros to speed up compilation. One crate that uses watt is momo, in an effort to reduce compile times. (The source code of momo is still available, it ships wasm binaries to crates.io to make compilation faster)
The author of watt, David Tolnay, also tried to ship precompiled binaries with serde, to speed up compile times. However there was a backlash and it was reversed
I think that rustup should distribute precompiled versions of popular libraries, beginning with syn. It's really annoying that only the stdlib (well, std, alloc and core) is distributed through rustup in a precompiled form.
Then, when building a package that depends on syn, if you have it installed through rustup (rustup component add syn) and if it's the right version, you get to use it without compiling. syn artifacts are also not saved in the target/ directory
fun fact, this is actually not true! if you enable the rustc_internal sysroot crates (eg. rustc_driver) you'll see they actually include every dependancy of rustc and std, like libc, cfg_if, and hashbrown.
Does this mean that if I depend on hashbrown in the right version that is included by the stdlib, I get to use precompiled hashbrown rather than compile it from source? That would be amazing
No, and even if it was possible for cargo to detect an exact match and use the precompiled version in that case, the precompiled hashbrown is compiled with the rustc-dep-of-std feature enabled, so it wouldn't be eligible for replacement anyway.
Personally I like going the opposite way, -Zbuild-std should be the default approach and there should be no need to distribute any precompiled library assets.
I feel like I am missing some part of this discussion. How much of the request is just that it would be nice if cargo also hosted a shared/blessed compilation cache. This would be a definite QoL improvement, and a cache allows for very straightforward paths for missing the cache (my platform/whatever isn't in the cache, I guess I'll compile it myself).
Shipping precompiled artifacts implies some work on the packager (not cargo), and also implies it's some kind of error, as opposed to a miss, if the artifact isn't present. But if this whole discussion is just that there isn't a nice shared cache, so we would like cargo to support people sort of making their own, then maybe we should just make the cache thing work. Potentially only a small whitelist of crates would be cached, but that would still be a huge improvement.
This would also address almost all of the reasons why I'm not comfortable with individual packagers bundling precompiled artifacts, provided that it was hosted on trusted infrastructure and served only 100% deterministic and reproducible derivations from the source packages. Like Nix binary caches / Guix substitute servers.