Merging core, alloc, and std is something that has been thrown around on a number of occasions and has clear benefits (such as eliminating coherence hacks in the compiler among other things).
What would be the process to actually begin doing this? I assume (naïvely) that it would involve creating alloc and std features inside of core and moving things over piecemeal. However, I have not the slightest clue how that would work with packaging things up for a toolchain release.
Perhaps more importantly, is this something teams are interested in? It's a cross-cutting concern, presumably involving libs (duh), lang (coherence), compiler (rustc_ attrs), and release (packaging).
Strongly related: how could a situation such as core::panic::PanicInfo and std::panic::PanicInfo be handled? The two are surprisingly different types, which is something pointed out to me when I raised this in passing previously.
It's still something libs is interested in doing. I think the best next step would be putting it on a libs meeting agenda (file an issue and libs-nominate it). We can talk through how we want to go about trying this as an experiment, and any blockers. Assuming we're prepared to go forward with it based on that, we can continue talking about the design specifics async on Zulip.
I expect we could handle the PanicInfo difference using feature flags, such that just-core gets one thing and std gets both with different paths. I can imagine a couple of ways to do that.
I don't see how we can merge core, alloc and std without breaking changes. The precompiled standard library has to be compiled with alloc and std support enabled on most targets, but that would break #![no_std] programs that depend on libc not getting referenced or that use #[panic_handler]. Even just merging core and alloc would break #[panic_handler] without -Zbuild-std and mandating -Zbuild-std would break pretty much everyone who doesn't use cargo (and even cargo still wouldn't know whether to use -Zbuild-std=core or -Zbuild-std=core,alloc for a given program).
My first guess is that we might need to have more than one pre-compiled library, or otherwise do something similar to what we already have to handle the split.
This experiment does not necessarily need to immediately start presenting things differently to the end user. It can start by trying to present an identical interface with a different underlying implementation.
But how would rustc know which to use when compiling the dependency? Because multiple precompiled standard libraries are not ABI compatible, rustc would need to make a guess which one the user wants in the end when compiling a dependency, but if that guess was wrong, compiling the leaf crate will give an error about duplicate lang items.
As a user, a better solution would be to improve coherence so that STD doesn't need to have special hacks for it. I would rather see the ability to (soundly) define sets of crates that are in the same coherence domain, or something like that. This would help immensely outside std as well.
Furthermore, I want std to long term not be special. Yes, some parts in core do need special things that cannot be stabilised. But ideally everything alloc and std does should be possible for a normal crate to do.
This was my assumption as to how things would happen (hence marking the thread "internals"). The end user shouldn't care about this, at least for the time being.
core: --no-default-features
alloc: --no-default-features --features alloc
std: (default)
All three would remain surfaced as-is to the end user, no guessing necessary. Further down the line we could potentially eliminate the distinction altogether.
Agreed, but I don't see any way to do that soundly given that crate names are user-defined.
The problem with this approach is that today, you don’t know whether a given crate depends on std until you start compiling it, so it’s not possible for Cargo to pick the right set of std features up front. So, this can’t be done until everyone migrates to a world where Cargo.toml includes std dependencies.
Suppose that crate foo depends on crates bar and baz, and crate baz is #![no_std], but none of the others are. Then the build process might proceed as:
Compile bar, linked against full-feature std
Compile baz, linked against no-features std
Compile foo, linked against bar and baz ... oops, we have a conflict between two stds.
For today’s Cargo features, Cargo solves this problem by computing the features to activate based on [dependencies] info. But there is no such info for no_std — it can only be discovered by actually performing the build.
If coherence domains are created by passing the same value to -Ccoherence-domain and in addition you need an extra rustc invocation after compiling all crates in the coherence domain that you pass all members of the coherence domain to for checking the coherence of all crates combined, producing a proof file, then it wouldn't be possible for users to cause conflicts. Cargo would give each coherence domain a unique name and in addition make sure no unintended crates end up in it by not passing them to the coherence check rustc invocation.
That can approximately never happen, as you can keep using crates on old editions and made for old Rust versions. Which means this proposal is dead in the water from what I can tell (unless someone finds a solution by really thinking out of the box).
The only thing that would affect is lang items, no? The compiler already handles re-exports of items from core in std, why wouldn't it be able to handle two identical definitions of lang items? Sure, there would need to be a way to declare them as identical, but that seems quite easy.
