Idea: Edition visibility for the standard library

[Centril]

Say for example that we would wish, hypothetically, to make uninitialized not just deprecated in edition 2021, but that we would like to make it go away entirely. Hitherto there hasn’t been any way for the standard library to make such changes because there can only be one standard library and each edition must use it. Thus, it would not be possible to use #[cfg(..)] for these means.

@eddyb has raised the interesting idea of having a sort of “edition visibility” with which you may hide things from other editions. The straw syntax is pub(2018) meaning that this is only visible in edition <= 2018.

For example, you could say:

#[inline]
#[stable(feature = "rust1", since = "1.0.0")]
pub(2018) unsafe fn uninitialized<T>() -> T {
    intrinsics::uninit()
}

Another case in which this could be useful is:

pub(2015) macro try { .. }

This would make the try!(..) go bye bye in Rust 2018.

Of course, if you use a <= 2018 crate, then you can still indirectly reach uninitialized but the goal here is to make it highly unlikely to be used.

I believe it should also be possible (please do double check my reasoning here because I am somewhat unsure…) to change the type signature of a libstd inherent method by doing something like (we would never actually do this with Vec::new):

impl<T> Vec<T> {
    pub(<=2018) const fn new() -> Vec<T>  { .. }

    pub(>2018) const fn new() -> (Vec<T>, usize) { .. }
}

However, this is not the likely use case.

@eddyb also floated the idea of having an unstabilization mechanism but said that visibility is a stronger method.

One nice thing about pub(2015) is that we may be able to do this in an unstable way and never stabilize it if we want to. Thus, it may be a mechanism that only the standard library gets to use and so we could even use this for edition 2018 but delay any bikeshedding for quite a bit of time.

For the bikeshed, @kennytm suggested that the syntax

pub(<= 2015)

could be clearer with respect to intent.

@kennytm also noted that the syntax should try to highlight that it is the visibility of the downstream crate and not the defining (current) crate.

[alercah]

I really like this idea, but I want to throw something else on top too: why limit this to the standard library, and why limit it to visibility?

The fundamental problems that editions try to solve are far from unique to the language. Especially where traits (Error, I’m looking at you) are involved, there is benefit to being able to clean up an API in a breaking way, while retaining ABI compatibility so that you don’t cause a massive ecosystem issue. The non-stdlib crate that comes immediately to mind is serde: serde 2.0 is about as viable as Rust 2.0 at this point.

While the exact mechanics could be discussed (for the record, I completely support doing this with try! in 2018 with some unstable mechanism until a more detailed proposal can be developed), this could be a powerful feature to have a great ecosystem that can evolve naturally in the same way that the language hopes to.

[Centril]

@alercah Any ideas on how we could make this happen? ABI compatibility is tricky because we don’t want to tie our hands wrt. optimizations and letting the language evolve. If we can do it nicely without too many drawbacks I’m all for that.

Limitations for libstd are merely suggested here to solve the immediate problems ^.^

[alercah]

We don’t have to make separate version of the library ABI-compatible, we merely need to make separate “editions” of a library within the same version ABI-compatible when compiled with the same compiler. So in your example with Vec<T>::new, the compiler might mangle them about differently.

A more interesting example might be RFC 2504, where fundamentally, we want to change the return type of cause. We can’t do this in a backwards-compatible way, but what if we could? Something like the following, perhaps:

trait Error: Display + Debug {
    fn backtrace(&self) -> Option<&Backtrace> {
        None
    }

    pub(>= 2018) fn cause(&self) -> Option<&dyn Error + 'static> {
        None
    }

    #[deprecated]
    pub(<= 2015) fn cause(&self) -> Option<&dyn Error> {
        <Self as Error@2018>::cause(self)
    }

    #[deprecated]
    pub(<= 2015) fn description(&self) -> &str { "an error happened" }
}

#[deprecated]
fn non_static_cause<E: Error>(e: &E) -> Option<&dyn Error> {
    if edition!(E as Error, <= 2015) {
        e.cause()
    } else {
        e.cause().map(|e| e)
    }
}

The ABI treats the two cause functions as different, but the surface would be different. In many cases when you want to change trait APIs, it may be possible to go in both directions so you wouldn’t need the extra non_static_cause helper to get around the edition issues.

Of course, there would still be some impedence issues. In this particular example, you’d have to put a big red warning on the package that doing an upgrade will require you to inspect uses of cause to see if you need to switch to non_static_cause instead. But this would allow you to actually remove the deprecated version of cause from the API exposed to programmers to implement. It would be especially useful in situations where a future version of a trait doesn’t change the functionality, so that the old and new APIs are implementable in terms of each other. In that case, you could just change the API, and it would Just Work across libraries in the same program using different editions of the crate.

(something something maybe “edition” isn’t the best choice of term)

[nikomatsakis]

@alercah

I really like this idea, but I want to throw something else on top too: why limit this to the standard library, and why limit it to visibility?

This seems quite related to the idea of extending unstable/stable "channels" to arbitrary packages -- e.g., I think that Rayon would like to export unstable APIs, but have them by "infectious", so that downstream users do not casually rely on them (in particular we don't want it to be possible for you to "hide" the unstable APIs Rayon by wrapping them in another crate). We achieve this now with a custom cfg.

[repax]

Could this be a possible alternative?

#[edition <= 2018]

Or if not that then:

#[cfg(edition <= 2018)]

[Centril]

@repax

Could this be a possible alternative?

Sure; That could work. Presumably it would just desugar to some internal visibility mechanism?

(I'm mostly interested about the semantics rather than the syntactic aspect at this stage)

@repax

Or if not that then:

The problem here is that all crate with some edition link to the same standard library. This means that all items must be present in every edition and you can't conditionally compile them out.

[mark-i-m]

Related work: This is how go solves the problem: https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md

Personally, I like @repax’s solution best. It basically allows us to create editions of crates, just like the compiler.

[binomial0]

I see the advantages of this idea, but we should keep in mind one serious drawback: this would make upgrading your edition a nontrivial task.
At the moment, the edition changes are purely syntactical and can (afaik) be done by rustfix. This makes upgrading the edition trivial, which is a major selling point of Rust 2018.

If we remove mem::uninitialized in Rust 2018, there is at least a functionally-equivalent alternative. This might be upgraded by rustfix in this particular case, but in general, I believe it will be quite hard to come up with automatically-applicable fixes for every deprecation.

I also fear that later, we might want to remove a part of the language that turned out to be a bad idea, without offering a direct, functionally equivalent replacement. (This is an even bigger issue if we open up this feature to library authors.) When this happens, users will have to redesign parts of their software to upgrade to a new edition. This contradicts the promise of stability that Rust is making, and it would also be a pity if some users missed out on the new edition features like async syntax because they cannot / don’t want to rewrite parts of their crate.

One solution for this would be to allow users to opt into older editions locally, maybe using the syntax Identifier@2015 that was introduced above. This would be similar to deprecation, only with a deny-by-default lint.

[Centril]

@binomial0

If we remove mem::uninitialized in Rust 2018, there is at least a functionally-equivalent alternative. This might be upgraded by rustfix in this particular case, but in general, I believe it will be quite hard to come up with automatically-applicable fixes for every deprecation.

I don't think there's a plan to remove mem::uininitialized in Rust 2018 and we wouldn't do this until there is a better alternative.

In the case of try!(expr)?, we would rustfix folks in favor of expr?.

@binomial0

I also fear that later, we might want to remove a part of the language that turned out to be a bad idea, without offering a direct, functionally equivalent replacement.

I think as a general principle, we would make removal conditional upon having a rustfixable mechanism that works well in almost all cases.

@binomial0

(This is an even bigger issue if we open up this feature to library authors.)

Yeah; that's a problem. I guess we would need to have some mechanism for library authors to tailor rustfix to their crate so that users can automatically transition from one major version of a crate to another.

[CAD97]

It’s been proposed before, but this would be a key use point for #[deprecated(replace_with = ...)].

Interesting quick/unbaked idea: write a macro-by-example replacement mapping from the old deprecated and hidden, rustfix will then pick up that deprecation as machine-fixable.

[binomial0]

@Centril

I think as a general principle, we would make removal conditional upon having a rustfixable mechanism that works well in almost all cases.

This is good to hear. I think it would be a good idea to explicitly state that "rustfixability" is a hard requirement for any such edition removals. Removing an item without a working rustfix fix is a breaking change, and should be treated as such.

I don't know what you mean by "in almost all cases". I think we should apply the same standards to this as we do now to breaking changes. We should only allow a fix to fail if it is unlikely that even a small number of users are negatively affected at all.

[Ixrec]

@binomial0

This is good to hear. I think it would be a good idea to explicitly state that “rustfixability” is a hard requirement for any such edition removals. Removing an item without a working rustfix fix is a breaking change, and should be treated as such.

For what it's worth, we more or less agreed on that point about a year ago in Committing to a Rustfix Tool for All Epochs/Checkpoints Changes, and this section of the epochs RFC explicitly states that consensus in far more precise terms.