Version-based visibility to make the language&ecosystem more stable

I found two sources of potential API instability that could be solved with similar approach.

Case 1: added trait method causes breakage

// in crate A
trait Foo {
    fn do_stuff(&self);
}

// in crate B
trait Bar {
    fn do_something_else(&self);
}

// in crate C, which depends on A and B
use a::Foo;
use b::Bar;

x.do_stuff();
x.do_something_else();

Now if either A adds default fn Foo::do_something_else or B adds fn Bar::do_stuff C breaks due to ambiguity. This is considered minor and can be resolved using UFCS however the breakage is still annoying and UFCS is ugly.

This is even worse for core where e.g. TryFrom/TryInto could not have been added to prelude in old editions because people wrote their own traits to work on stable. Now people have to jump whole edition to have it in prelude. Since core is used by everything the likelihood of breakage is high.

Case 2: functionality got moved to a feature

Crate A has a default feature foo which is undesirable in crate B, so crate B specifies default-features = false } Crate A wants to make some dependency optional but this dependency was previously used by B. Making the dependency optional but default doesn't work, so the only solution is for A to issue breaking release.

It seems this is most often experienced when crates make std optional but there are other cases too.

Additional annoyance

I work with the newest compiler to have better error messages etc but want to support older MSRV. After I write the code I test with older MSRV and find out I have to rewrite it because I misremembered which features are available in the given MSRV.

This is unrelated but happens to be helped with by the solution!

Solution

The solution for both would be to make the new crate versions somehow act as if they were older.

For case 1

Add attribute #[since = "$version"] to newly added items in the dependency (A, B in the example). Then if a consumer crate (C in the example) specifies older version of dependency in its Cargo.toml those items become invisible. Attempt to use them will cause compilation error with helpful message suggesting to bump the version. In case of std&co stable attribute is reused and combined with rust-version field in Cargo.toml. This behavior is only enabled with option hide-unrequested-features = true (feel free to bikeshed) to avoid breaking existing things.

This helps with the annoyance above at least when it comes to libs API.

For case 2

Add optional default-before field to features:

[features]
foo = { default-before = "1.4", deps = ["bar"] }

If a crate depends on version 1.3 it will see foo feature as always on even when default-features is false. To truly remove it the crate has to depend on 1.4 or higher.

I imagine this could also reduce many discussions around "this is minor change but let's do crater run; oh breaks x crates, is it too many?" and enable faster library development.

Can you see any way this could be improved even more? Would it be worth adding?

6 Likes

I would love the integration of version and function resolution. Better backwards compatibility and a mechanism for actually utilizing the rust-version attribute for validating against the feature-set are both good reasons on their own.

The biggest downside is that same story, though. This silences some forward compatibility issues but doesn't help resolve them. Bumping rust-version to larger versions is something that should be trivial. However, this feature allows std to silently become incompatible with such upgrades. Consider the motivating example of an added TryInto. We've moved the point of failure from the release of Rust 1.70 to the bump to 1.70 in the crate. A compilation breakage that could have been previously considered to cause too much ecosystem breakage could slip through the current crater checks—afterall all the crates which pin their compatibilty still work? But it will take the exact same total toll and dev churn when the ecosystem tries to upgrade beyond the rust version that now contains the changes, the effect is just delayed and stretched to a longer timeframe.

Yeah, I was thinking about forward compatibility a bit. I think the lint should still be available but I'm not sure if it should be allow or warn by default.

Add attribute #[since = "$version"] to newly added items in the dependency

What I'd like to see

  • Expose the stable attribute used in the stdlib
  • Warn if an item is newer than the bottom of the version requirement (or rust-version for stdlib)

Add optional default-before field to features:

There have been several directions people have gone with migrating features

default-before proposal is similar to RFV 3146 in that it makes decisions on behavior based on your version req. I'm a bit concerned that changing a version req to a newer, compatible version would be breaking someone's code.

1 Like

One question I have is: what happens when resolution ends up forcing something newer anyways?

Say you have crate A that does this and adds some symbol X with #[since = "1.4"]. This conflicts with crate B's X in some way, but it has a = "^1.3". Crate C uses X and has a = "^1.4". When compiling A, is X included or not? If it is not, C fails. If it is, B needs some way to hide certain symbols when looking into A based on its requirements (this would require some more information on the command line when using A's .rlib).

Whatever the solution, I suspect this needs a Cargo.lock update to say something like "a 1.4.0 (as of 1.3.0)" or something in the dependency list (obvious bikeshedding opportunity, but I think this info would then be required in the file somewhere).

Great that people are thinking about those! Will check them out.

I'm a bit concerned that changing a version req to a newer, compatible version would be breaking someone's code.

How could it? If you change it for your own crate you may get compile error but that's because you made an explicit change. You have to either update the code or roll back. Nothing else is affected.

In your example B doesn't see the symbol X but C does. It could have interesting interactions with re-exports but I think they are solvable. For * imports just exclude the new items, for added struct fields, enum variants and (trait) methods just make them inaccessible from C when going through B but accessible directly. Actually, I think only * imports are an issue and excluding looks simple.

B needs some way to hide certain symbols when looking into A based on its requirements (this would require some more information on the command line when using A's .rlib)

Yep, adding an argument for this seems fine to me.

Cargo.lock update

Does cargo guarantee it only reads Cargo.lock in some circumstances? I don't remember such guarantee and if it doesn't it can just read Cargo.toml to get the required information.

As of today, upgrading version reqs that are semver compatible is a safe operation (pending regressions). I am very loathe for us to deviate from that.

Besides just the breaking, the rest of the story isn't that great. The upgrade instructions need to be inferred from reading your dependency's Cargo.toml. We could make cargo upgrade handle that for you but you shouldn't have to use the tool to do upgrades and you shouldn't have to dig into source or even changelogs to upgrade within semver compatible versions.

Upgrading being "safe" depends entirely on whether the dependency copied its policy from std ("A bunch of really smart people behind std chose this policy so it must be good, right?"). Per Rust policy adding a default method to a trait is minor change so breakage absolutely can already happen! Even worse, it doesn't even require Cargo.toml update, it could be update of Cargo.lock out of habit or because one wants to use new stuff in an unrelated crate that depends on the same crate. My proposal strictly improves this situation by limiting the breakage to explicit Cargo.toml update.

Updating Cargo.toml should be only done when one actually needs to use a feature. Updating it without a reason is harmful. If one's going to touch the code to use a new feature anyway, resolving the breakage should not be a big issue. Note that these breakages are very rare anyway and also trivial to fix: just change call to UFCS. Using an official tool doesn't seem bad, cargo itself is an official tool.

Of course there might be crates that hate breakages and would bump major even in case of added method. I strongly believe this would be much worse than changing a few calls. I already experience major PITA when dealing with dependencies that change major versions. Libcpocalypse comes to mind as an infamous example of widespread breakage. If the language can be modified to reduce these cases even imperfectly it's much better than the alternative.

Finally, I think that additions to std not being breaking at all (at lesat if you have rust-version specified) could be a great additional argument towards convincing Linux distros to make an exception for Rust and update a bit faster (they could even make rust-version mandatory in their own packages to make it nearly perfect). Given recent MSRV discussions, this is also great and worth it.

I don't see any reason this would need to be in the Cargo.lock. It seems to be possible simply as an additional flag giving the version requirement of all dependencies to the crate when it's building.

I assume this is going to only be supported when using a simple ^ version requirement? It likely isn't possible for #[stable] attributes to work correctly across major versions if you have a version requirement like > 1.1.2, < 3.0.0

1 Like

The likelihood of a minor incompatibility breaking someone is relatively low such that people regularly update all dependencies.

As for not touching Cargo.toml, you might want to catch up on the cargo upgrade discussion as we are looking to make cargo update --save to update all version requirements.

How much would this be mitigated by testing crates, whether in their CI or in Crater, both in an MSRV mode and in a mode that uses latest stable or newer?

For > x, < y supplying x should be correct.

@8573 there would have to be some special lint level and you'd have to set it properly but it should completely resolve the issue.