Distribution of rustc libraries

tools and infrastructure
#1

Currently we distribute a shared library for each internal rustc library we have. However, ultimately, what really needs to be exposed are two symbols main for rustc and main for rustdoc; everything else may be as private or internal as they need to be.

High level proposal is to instead ship a large librustc_everything.so/rustc_everything.dll/etc shared library and two ultra-thin binaries which essentially are

#[link="rustc_everything"]
extern "Rust" { fn rustc_main(); }
fn main() {
    rustc_main();
}

rustc_everything would link all the libraries we currently ship statically and only export the two entry points for rustc and rustdoc (potentially others for whatever other tools we would produce).

This proposal, would help us with enforcing our stability story. Currently you can link to the librustc_* libraries with plain -l circumventing any stability checks and thus increasingly de facto stabilising crates we do not want to stabilise (most notably I myself experimented heavily in that direction with the llvm_build_utils crate and librustc_llvm library). It allows us to hide stuff like rustc-internal serialize or getopts as well.

That being said, what I’m proposing here is obviously a breaking change. It breaks anybody linking to rustc crates via -l. And it breaks anybody using extern crate getopts without cargo and with the unstable(rustc_private).

I’m still not entirely sure how this would work with syntax extensions, though. Do they need (some) internal crates exposed to have what to link to? At runtime they obviously could just use what’s already loaded into the process space, but what about compile time? We could have rustc_stuff_for_syntax_extensions and rustc_everything_else, or rustc_everything could selectively re-export crates necessary for syntax extensions.

cc @alexcrichton @brson

1 Like
#2

If we were to ship everything in one binary I’d actually prefer to take the rustup route of shipping the same binary hard linked into two locations, and then we change the behavior on how it’s invoked. In any case though the compiler supports plugins today which ties our hands in terms of how we ship these libraries.

Because any plugin can link to any subset of compiler libraries, we basically have to ship dlls for all of them :frowning:

#3

Even though plugins can access arbitrary public functions from any library we ship there may still be good reasons to combine all of them into one uber-dll (though I’m not sure what they might be). It’s also very likely that extant plugins only use a small subset of the crates we distribute, so we could pursue a strategy of locking down the ones that aren’t in use to prevent further de-facto stabilization.

On that last point, just a slightly different idea: we could find the set of rustc-private libraries that are basically unused in the ecosystem and give them a new attribute that requires the bootstrap key to load.

#4

cc @nrc some discussion here about reducing plugins’ access to certain private crates.

#5

I think we have to at least let syntax extensions have access to libsyntax until we have a proper procedural macros system in place. This seems like it would affect tools too - they would no longer have access to the compiler internals. While I would like to lock the compiler down for stability reasons, it seems that this would curtail a lot of useful experimentation work.

Is there evidence of de facto stabilisation becoming a problem? Would this plan have any other benefits?

1 Like
#6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.