Thoughts on aggressive deprecation in libstd

libs
1

I’ve been thinking recently about the policy of deprecating stable APIs in the standard library recently, and it looks like we’re about to accept an RFC which is paving the way forward for future decisions. As such, I just want to make sure that we’re all aware of what the repercussions may be! If you’re aware of the discussion on the size_hint => len_hint RFC, you can skip the next background section

Background

We currently have an RFC to rename the size_hint method on the Iterator trait to len_hint to make it more consistent with the term len used for other collections. In a vacuum this RFC appears to have fairly broad consensus that it is the right decision to make, but we’re in a post-1.0 world where we need deprecation paths forward and can’t simply just remove size_hint.

The proposed solution is to introduce len_hint as #[stable] while simultaneously deprecating size_hint. The deprecations will not be removed until Rust 2.0 and hence all existing code will continue to work other than just receiving compilation warnings.

Why deprecate APIs?

The standard library at 1.0 has a considerable size, and it inevitably has a few minor inconsistencies in naming and other little aspects here and there (as seen with this recent RFC). As a result, we can either aggressively try to fix these inconsistencies, or we can live with them over time.

I personally would like to be able to fix all of these inconsistencies as they come up over time as it has a number of benefits:

  • All future newbies will have a better experience with a more consistent language
  • All existing Rust programs will continue to work, they’ll just get some form of warning about the newer form (which can be silenced)
  • This sets a strong precedent for “the standard library is not stuck”, which I think is a huge boon for the development of the library over time. I don’t want to be wed to old APIs when small backwards-compatible changes can be made to slowly improve the experience over time.

There are, of course, a number of downsides to deprecating APIs

  • A deprecation can be seen as a breaking change for one of two reasons:
    • Migrating away from a deprecated API means giving up support for older versions of Rust as the new version likely doesn’t have the deprecated-for replacement.
    • If one aggressively does not want warnings in a build. Deprecating an API sends a strong message that callers should change, which can sometimes provide unnecessary churn in the community (e.g. downstream crates also decide to change their APIs).
  • If the standard library evolves radically over time, old code snippets seen, for example, on Stack Overflow will get out of date and may be entirely obsolete at some point in the future (riddled with deprecation warnings)
  • Most deprecations are for minor features, such as the naming of an API, and it’s tough to see how it’s worth it for the costs above.

I would, however, like to proposed that the standard library should aggressively deprecate APIs and continue to evolve. This puts it in a great position to keep up with idiomatic Rust while maintaining backwards compatibility. I think we can take at least one measure to address the first con as well as the last con, but the second one may be fundamentally insurmountable.

Supporting old Rust versions

A major problem here is that updated code will no longer be able to compile against older versions of Rust, which can sometimes be quite important. It’s unclear to me whether this will actually be a problem in practice as everyone may stay up-to-date as much as possible anyway, but I think it’s at least worth considering.

The current story here today would be to slap #![allow(deprecated)] at the top of a program and call it a day (after avoiding all new APIs), but this is not an ideal situation as there may be deprecations you want to know about and can replace with crates.io crates, for example. In the “ideal world” I would like to see the ability to do something like:

#![allow(deprecated(foo, bar))]

Here, only deprecation warnings for the features foo and bar are allowed, otherwise all deprecation warnings are printed. We would ensure that each deprecated API has an appropriate feature name (e.g. not rust1) so it could be listed in the deprecation warning.

I believe a system like this (whitelisting deprecation warnings based on features) will easily allow crates to maintain backwards compatibility with older Rust versions. The list inside the deprecated attribute I don’t think will grow without bound because over time if it’s too large you’ll just drop support for older Rust versions (which are likely super old by that point).

Immediate actions

In the near future, I would personally like to accept RFC 1034 assuming that a system (not the exact one, but similar to) like described above is implemented. I don’t think the RFC needs to be blocked on fleshing the system out entirely, but I do think it’s important to “just accept” the RFC without considering the implications of deprecating standard APIs. We’re going to be setting a precedent if we accept the RFC, and just want to make sure we’ve got all our ducks in a row!

How do others feel about this course of action? Specifically:

  • Do you agree that the standard library should aggressively deprecate APIs for consistency?
  • Do you think the ability of supporting older Rust versions is important?
  • Do you think that #[allow(deprecated)] is sufficient, or do you think a more fine-grained control will be required?
  • Can you forsee other problems with deprecating APIs in the standard library?
13 Likes
2

I think we’re over similar minds here, with perhaps the caveat that I’m more skeptical of people ever updating their code. I think that if we agree that deprecations should be a thing to not fear, then we should take a “deprecate early” stance, so that the ever mythical 2.0 will be less of a system shock when it happens. It won’t be a period of massive deprecation and removal. It will simply be The Day The Deprecated APIs Finally Went Dark.

This of course relates to our story on how rustdoc renders deprecated APIs. If they don’t show up, that could cause serious confusion! Especially since Rustdoc already needs to work very hard to “unwind” certain magic like Traits and Deref that pull methods from the aether (and still fails in a few places).

I’m personally favouring the idea that as the x in 1.x.y grows, we become increasingly shy towards deprecations. The size_hint -> len_hint change seems slightly more reasonable to me today since 1.0 is mere weeks old. But if we were at 1.10, I’m not sure I’d have the same opinion!

Lots of busted Rust code on the internet is something we already struggle with from the pre 1.0 days. I’m probably a bit desensitized to it.

Edit: also the forward-compat (or whatever it’s called) concerns seem particularly salient. Forcing Rust upgrades because a library didn’t want to put #[allow(deprecated)] is a bit nasty.

4 Likes
3

Yes!

No. I think renaming is a weak reason. Has it ever shown up as a point of confusion for newcomers to the language? And has it ever led to confusion that produced incorrect or even dangerous code? I estimate both are negative.

I think the gains in this particular case are not clear, and I don't think that renaming something that doesn't touch on the critical issues just mentioned, it doesn't deserve the effort it takes to cope with deprecations from both users & developers.

2 Likes
4

I agree with Gankro, the bar for deprecations should increase with time. Deprecations for really good technical reasons (say, because it’s really wrong, or superseded by a much better alternative) should always be pursued aggressively, but small inconsistencies in naming… meh. Everyone deals with sub-par names all the time. After two years, the churn may not be worth it. But maybe this should be relative to when the specific feature being deprecated was introduced? Renaming a thing that’s been around for ten versions is frivolous, but introducing it in 1.x and renaming it in 1.(x+1) causes less pain. Still looks unprofessional, but so does inconsistent naming, and RFC process + code review + trains should catch most of these before they get into stable.

Regarding lints: I like the sound of features for allow(deprecated), but the main advantage is that you can hold onto specific deprecated pieces without letting the rest of your code bitrot, and that can mostly be achieved with #[allow(deprecated)] on the right items. Maybe if #[allow(deprecated)] use std::foo:Foo; disabled all warnings for Foo in this module?

4 Likes
5

I’m broadly in favour of aggressive deprecation.

One thing we should be aware of is that Rust has had a reputation (due to being developed in the open) as a rapidly breaking language. The big fanfare over 1.0 has remedied that to some extent, however, if we are seen as a language with lots of deprecation, then we might recover that reputation, which would be a shame.

I think this should not mean we don’t do it, but that we should keep in mind how we market these things.

4 Likes
6

Yes.

Yes, though not a blocker.

Fine-grained control is always nice. Without it, it's just so binary: as soon as I decide one deprecation is okay, I miss out on all the other ones I may care about.

One question is what to do about docs. If we start to get too crufty, there's an thought that maybe we should hide deprecated items, but then it's hard to discover what old code does.

1 Like
7

The same is true for using "new" APIs. Code that uses socket timeouts will not compile on Rust < 1.2, for instance.

8

An idea that I’ve kicked around before to prevent annoying deprecation warnings is to annotate crates with #![rust = "1.1.0"]. Since stability attributes already include the Rust release the API was introduced, deprecation warnings can be omitted if the deprecation version > rust attribute version.

10 Likes
9

Yeah I definitely agree that rustdoc will probably need some love over time. Right now I don't think there's a great story for hide-by-default APIs, but my thoughts here are to show all deprecated apis for N versions, and then once it's N versions old rustdoc just doesn't render it at all. I suspect N could be ~4 to allow deprecated APIs to stick around in docs for 4 months, and after that they just disappear in all but the code.

We could, of course, add more fanciful facilities for hiding APIs, however, in which case this'd just be a "hide by default" after N versions instead of "don't render at all".

I do think it's tough to make a blanket "this is worth it" or not statement here. For example if BTreeMap::get did not have the same name as HashMap::get, I would very much want to rename the APIs as it saves me a trip to the docs every time I use BTreeMap. Something like size_hint in this specific case, however, is somewhat more ambiguous as it's rarely used and it could just be "one of those things" which you memorize instead of following a pattern to get.

I think this is a super important point as well! Just because we're deprecating something does not mean it's free, there are still costs associated with it. For example deprecating the name std in favor of something else would be a little... drastic!

Right, but I think this is somewhat orthogonal. Taken to the extreme this means we'll never add a new feature to the standard library, which I don't think is infeasible. As a library author I would like to have my crates work as far back as possible, but if I use a new feature from Rust I don't mind at all requiring a newer Rust version to compile.

Yeah I've thought of this in the past as well, but the problem I've found here is that it's still a very binary distinction. For example, in the set of APIs deprecated in 1.x, some could be renamings, some could be for bad performance, and some could be in favor of a higher quality crates.io crate (possibly). In this case I only want to opt-out of the renamings warnings, but I still want to deal with bad performance and using higher quality implementations.

1 Like
10

If I want to be compatible with Rust 1.x.y because that is what Debian packages or my build/release automation uses at the moment, I don't care what reason the deprecations have. There are good reasons to be compatible with older Rust releases even when developing with a newer compiler. The same reasoning applies to using newer versions as well. Conditional compilation keyed on Rust version could help with efficiency issues etc., but it's not everything.

2 Likes
11

Everything’s a trade-off but I hope deprecating for minor name inconsistency will be a really rare thing. You don’t want a naming situation like PHP has, of course, but stuff like size_hint doesn’t seem like a big deal & in the future when Rust has been stable a while that change would probably be very annoying.

Deprecation makes a lot more sense when its, for example, a new function with a “better” type signature, or abstracting commonalities of structs into traits & then deprecating some methods on the struct. I think those kinds of changes should be pursued aggressively.

1 Like
12

I worry that aggressive deprecation will not be worth the cost. Keeping Rust code up to date should not be a full-time job.

I’d be wary about changing names, except those that are obviously wrong. You should be able to support older versions of Rust without having spurious warnings about size_hint vs len_hint. It feels a bit like class and typename in C++ templates: it’ll just lead to inconsistency.

Broken or dangerous things should obviously be deprecated, but replacement APIs that are merely better need not force everyone using the old API to change their code.

13

This is pretty well known territory. I think the best approach is where the whole library is versioned (1.0, 1.1, 1.2), and application authors can say --target-libstd=1.0 and receive all deprecation warnings for < 1.0, but not APIs deprecated in 1.1.

Another bonus is that you can error if the caller tries to use an API that’s only available in 1.1. This will matter significantly more when Rust has a stable ABI.

7 Likes
14

I have no opinion on whether or not size_hint should be deprecated, but I generally do support an aggressive deprecation strategy.

However, I think it’s important that we seek to put a cap on the maximum number of deprecations that can happen in any one release (relative to the prior release). Giant walls of warnings only lead to people ignoring warnings altogether. We tread a fine line here. Given that Rust will have nine minor releases over the next year, I would favor a policy of spreading out deprecations when feasible.

I also agree that the bar for deprecations should rise over time.

At the moment it’s fine to assume that people will be upgrading their Rust code regularly as new minor releases appear. Rust is still a young language and people still remember well the speed required to keep up during alpha. However, as Rust matures, we should begin to assume that people will be stuck on old versions and perhaps try to accommodate them via a version attribute as proposed above.

This entire discussion is also an argument for a rustfix tool that would automatically clean up renamed APIs, as gofix once did.

15

I think it’s okay to aggressively deprecate things for the first few releases to have things as polished as possible going forward. This includes even minor renamings like the one discussed. In the future, I would expect, as other’s have mentioned, that the height of the bar to deprecate a feature would be proportional to the amount of time the feature has been stable. However, I think we should always deprecate features that have clearly superior alternatives, regardless of the features age.

If we end up with some kind of --rust-version=X.Y[.Z] option as originally proposed in #1122 before it was pared back, I would expect to only receive warnings for features deprecated before the specified version. Without the ability to specify a target version, I think specifying which feature deprecations to ignore is probably the best approach.

16

Looking at what other projects have done in the paste might be prudent. For instance, the Qt libraries have a much larger API surface than the Rust standard library; Qt has also been around for far longer (since 1991!). What Qt does is deprecate the old API and then undocument it too. No loud warnings are emitted when deprecated APIs are used (even though they could have been; most C++ compilers provide #pragma, #error and #warning directives that can be used for custom errors and warnings); old code just continues to compile while new code uses the new methods.

The old methods stick around for the life of a major release, which for Qt 4.x was seven years. The docs for the deprecated methods could actually be found, they were just tucked away into other parts of the documentation, away from the “main” docs for a given class.

To sum up:

  1. Aggressively deprecate old APIs. Code using them continues to work until the next major release.
  2. Do not by default emit warnings for the use of such deprecated APIs. Don’t create problems for people maintaining old code. It’s just the right thing to do; when those APIs were used, they were stable and recommended. The contract provided was “this will work at least until the next major release” without “…but we may hassle you about it in the future.”
  3. Make docs for deprecated APIs hard to find by default. Rust doc pages could have a settings checkbox for “show deprecated APIs” which is by default unchecked; the point is that it’s hard to use a deprecated API by mistake.
7 Likes
17

+1 let's not live with debt/annoyances forever (even if they're small).

Emphasis mine.

18

To expand on this, Python used to throw DeprecationWarnings when old APIs were used. This was turned off in Python 2.7 because everyone found it extremely annoying.

2 Likes
19

If we limit these warning to the root crate (not the dependencies) we will be just fine in my opinion. Most stuff will take < 5 mins to fix.

20

I’m just someone who saw this thread on reddit.

I like the idea of doing the deprecates by version number by crate/library. If my application needs to support stdlib >= 1.1 and libFoo >= 0.5, then I probably don’t want deprecation warnings for things that were depreciate in 1.2 and there’s no alternative in 1.1

By the same token, it should warn or outright error if I use stuff added after minimum version I want to support.

But there should probably be a “broken” flag that super deprecates something. Think of things like gets from the C stdlib.

This topic kind of brings up, at least in my mind, the topic of ABI.

I don’t recall seeing the topic mentioned in any documentation I’ve read so far, but it doesn’t look like Rust has a stable ABI. (Some googling finds me https://github.com/rust-lang/rfcs/issues/600)

Normally, at least in C world, you only want to break ABI compatibly with a major version release. I don’t know if adding a new method (in order to rename an old one) to a trait breaks ABI compatibly or not. But that’s something you’ll want to eventually document and then possibly worry about when changing the standard library, or any library really. (Or any application that has plugins.)

1 Like