Thoughts on aggressive deprecation in libstd

[jamesbaxter]

Do you agree that the standard library should aggressively deprecate APIs for consistency?

I strongly agree, especially for consistency, so I would highly be in favor of what people are calling frivolous to changing size_hint to len_hint. As someone mentioned earlier, if you don’t change it early on, you’ll end up with something like the monstrosity of an API that PHP created. I would much rather not live with debt.

A small number of people may cry of these changes early on, but I am willing to bet that many more will pin them as points of a rant if they’re kept as is.

Do you think the ability of supporting older Rust versions is important?

Between minor versions, yes, but not between major versions. Projects lock to particular compiler and library versions all the time.

A note about the docs: never hide anything on a documentation page. Use some css to gray it out, and add a note with the new function to use. That would be the most useful solution. Just please make is searchable with Ctrl+F. Apple’s documentation is one of the worst offenders of this, hiding everything by default, and it’s absolutely aggravating.

[emk]

The word that concerns me the most in this proposal is “aggressively.” I understand that sometimes APIs are truly awful, like the infamous case of creat in Unix, or the security issues of gets. An occasional deprecation of a exceptionally awful API is, I think, understandable. And at this early date, even size_hint seems reasonable enough to fix. (Update: The proposed fix means that len_hint would return incorrect values when applied to Rust 1.0-era iterators. This feels a lot closer to a breaking change to me.)

But at the same time, I just spent several weekends of my life fixing libraries that were broken by the push to 1.0. I’m exhausted from Rust churn and breakage, and I’ve finally been enjoying the idea that I can write code, and I can expect it to stay written without constantly revisiting it. I have the time to actually learn Rust for real, and to figure out good ways to accomplish common tasks. And most of my third party dependencies actually build on any given day. I finally feel confident that I can start to use Rust for real software.

Here’s how I imagine “aggressive” deprecations playing out:

1. Library X works perfectly well under Rust 1.5, and it is widely used by other libraries and by applications. Library X is mostly stable, but needs the occasional bug fix or minor enhancement.
2. Rust 1.6 deprecates a bunch of features used by library X, leading to massive warning spew every time that a user of library X tries to build it as a dependency.
3. The author of library X fixes all the warnings. But now their library only works with Rust 1.6, and it can no longer be built using the system Rust on Amazon Linux/RHEE/Ubuntu LTS 16.04.
4. If I want the latest bug fixes for library X, I need to start upgrading lots of other things.

As long as Rust keeps churning, it forces everybody to pay more attention to rustc and std, and less attention to interesting crates and applications. And I’d much rather live with an occasional odd API than spend my time dealing with cascading updates across the ecosystem.

So for occasional, careful deprecations of genuinely awful APIs. But for “aggressively” deprecating large swaths of the Rust API. My main concern here is that if the word “aggressively” is actually included in the official policy, it will be used as a rationale for a lot of API tinkering that privileges an ideal vision of the std library over actual Rust applications. The single best feature offered by infrastructure software is staying put, so other people can build around it.

[tshepang]

One question is what to do about docs. If we start to get too crufty, there’s an thought that maybe we should hide deprecated items, but then it’s hard to discover what old code does.

They can be hidden under an Expand button… “Show deprecated stuff”

[seanmonstar]

Rust 1.6 deprecates a bunch of features used by library X, leading to massive warning spew every time that a user of library X tries to build it as a dependency.

Currently, cargo builds all dependencies with allow(warnings), so you should never see the deprecation warnings from a dependency, only those you use in your own crate.

[tshepang]

To expand on this, Python used to throw DeprecationWarnings when old APIs were used. This was turned off in Python 2.7 because everyone found it extremely annoying.

The difference here is that these warnings are shown to end-users. In the case of Rust, only developers see them.

[tshepang]

But there should probably be a “broken” flag that super deprecates something. Think of things like gets from the C stdlib.

Breaking changes are allowed in 1.x for such bad APIs.

[shadowmint]

The point of 1.0 was backwards compatibility.

This basically breaks that guarantee.

I know, I get it, it’s will still compile with some versions, and it lets you cleanup stdlib apis which lets rust’s standard library not become python’s standard libray…

…but this is breaking the compatibility promise.

People write code, and then abandon it. It’s just a fact of life; there are already abandoned crates on crates.io.

What needs to happen is this: Crates that are published on crates.io always work.

Not… kind of work. Not, work with a particular revision of rust. They just work. That’s the compatibility promise.

So, if we do go down this road, we should also aggressively delist crates on crates.io that don’t for fill this promise.

(and if you want justification for this, look at pypi and python3. It’s a disaster. There are thousands of packages that will never get ported to python3, and still turn up in the first-hit for search results. We’re asking for the same problem here.)

[tomaka]

I don’t really see the point of #[allow(deprecated)]. The purpose of backward compatibility is that your code will continue to build without touching it. If someone modifies the code to add an attribute, why not just fix the deprecation in the first place?

I prefer a #![rust="1.0.0"], or an entry with a similar name in the Cargo.toml.

[tshepang]

How is deprecating stuff breaking backwards compatibility?

[shadowmint]

It no longer compiles with older versions of rust.

(ie. If a package is migrated to use the new non-depreciated api)

If you need the ‘latest’ version of rust to build a project in rust… well, you’re not much better off than we were with the nightly are we?

…that said, I suppose version pins would solve the problem for existing projects.

Really the issue I’m pointing out is:

1. Search crates.io
2. Find package
3. Install package
4. Package doesn’t work because it’s using an api newer than your rust version.
5. Repeatably step back in package versions until it works <— this is pretty rubbish

[td_]

(ie. If a package is migrated to use the new non-depreciated api)

This is not backwards compatibility, this is strong forwards compatibility of a kind Rust cannot possibly provide. The problem doesn’t come from deprecating old APIs, but introducing new ones. Anything relying on a new feature introduced in Rust 1.1 will not compile on Rust 1.0. The only way to guarantee that all code will always compile under every version of the Rust compiler is to have only one version of the Rust compiler.

The intention of Rust’s strong backwards compatibility guarantee is to encourage people staying on the head of the stable branch instead of using compilers which only compile a subset of the code the most recent version compiles.

[arthurprs]

If we limit these warning to the root crate (not the dependencies) we will be just fine in my opinion. Most stuff will take < 5 mins to fix.

As an example, if you build ProjectX you don’t get Deprecation Warnings from any of the dependencies. This way your project will compile just fine without any deprecations annoyances until 2.0. I don’t see any drawbacks in this approach.

No debt, no annoyances, no breakage, no need to define versions anywhere.

Am I missing something?

[DanielKeep]

I’m in favour of deprecating things aggressively.

I do think it would be really nice to be able to explicitly say to the compiler “I know X is deprecated, but I don’t care. It’s fine.” If someone looks at the deprecation warning and decides that they don’t have a problem with the name/potential slowness/weirdness that’s being fixed, they should be able to specifically accept that one thing.

It would also be nice if whatever mechanism this uses could be extended to user libraries in the future, but that’s likely out of scope.

[llogiq]

Having had 20 years of experience with Java – which after all is a wildly successful language – has shown that deprecation is a good thing for a language. Their strategy is one of aggressive depreciation without removing stuff unless absolutely necessary, so old code continues to work nonetheless.

I’d advise against hiding deprecated items in the documentation. I would rather see them greyed out and marked as deprecated. Perhaps moving them to their own section could reduce the clutter of the docs.

It may also be worthwile to have different modes of depreciation: Soft depreciation just marks stuff in the docs, but will generate no warnings – this would be for not-bad stuff like size_hint that needs not be removed. Hard depreciation would be for bad/confusing APIs once there is a better option (Edit: To clarify, I think that the compiler should unapologetically warn in those cases). At least the docs should show a clear update path to the new implementation; optimally, there could be a source-transforming tool that (semi-)automatically applies the changes (now that would be an interesting project).

Also like Java rust should strive to be backwards-compatible where it doesn’t hurt security. Having a compile-time target=<version> switch to select the target version (and thus disable warnings for things that weren’t deprecated in those versions) would be a good starting point (however, this may only make sense for major versions).

[ruudva]

Do you agree that the standard library should aggressively deprecate APIs for consistency?

When faced with a trade-off between doing the “right” thing and causing people trouble when upgrading, I would opt for the “right” thing. Hopefully Rust is going to be around for a long time, which means the amount of trouble a deprecation would cause is small compared to all the code that has yet to be written. If we don’t deprecate, all this future code would suffer from the annoyances.

A small example: C# opted for backwards compatibility instead. As a result, when implementing IList<T> you not only have to implement IEnumerable<T>, but also the non-generic IEnumerable (of which the enumerator returns objects that you have to cast). Even though generics have been available for 10 years now, all new code still pays the price of implementing things that are long superseded.

Do you think the ability of supporting older Rust versions is important?

For applications, no. For libraries, maybe. Releasing a new version of Rust does not affect the older versions. Updates are not forced on you, you can upgrade to a newer version of Rust when your are ready. The only problem is that new versions combine new features and deprecations, so if you need the new features, you get the deprecation warnings as well.

Do you think that #[allow(deprecated)] is sufficient, or do you think a more fine-grained control will be required?

Definitely more fine-grained; if I silence some deprecations now because I want to deal with them later on, I still want to be warned about other deprecations in new code I write.

[ArtemGr]

The real annoyance is not when you get a warning. It’s when you suddenly can’t build your crate because some dependency no longer builds on a fresh version of Rust.

Recently std::io::BufStream was feature-gated on nightly and that broke several crates (smtp, for example) and made it impossible to build a crate on both the nightly and the stable. This should never happen, if an API is stable it might be deprecated but it should compile NP until the next major release at least!

Also I think the deprecated items should stay in the docs to avoid the “wha?!” feeling.

As an example, if you build ProjectX you don’t get Deprecation Warnings from any of the dependencies. This way your project will compile just fine without any deprecations annoyances until 2.0. I don’t see any drawbacks in this approach.

Am I missing something?

Yes.

The only way to split compilation into several units (with cargo) is to split the project into multiple crates.

That means one often puts a lot of project code into subcrate dependencies and if their warnings are ignored then they will be missed by the developer even though he maintains all those subcrates.

[llogiq]

To expand on my idea,

• rustc keeps compatibility (via a --target compiler directive like Java) either forever (or at least for 2 major versions) unless there is a very good reason to remove a feature (e.g. security),
• cargo can have a per-crate configuration for target version that defaults to the latest available on the system
• Items that should be deprecated are marked with e.g. #[deprecated(since="1.1")] or #(deprecated_soft(since="1.2") (this is currently only valid code within std; we should allow library writers to also use it in their own crates – however the version constant would need to be the libraries’ instead of std's).

To reiterate:

• Deprecation is a sign of a healthy, actively maintained language and should not be seen as a bad thing. We should not hide it
• We should strive to maintain backwards compatibilty except for security/correctness concerns

[Nercury]

I aggree with @Gankro - in perfect world the 2.0 release would simply remove deprecated APIs, but they would continue to work up to that point.

Breaking backwards compatibility for a library is pretty bad, breaking backwards compatibility for a standard language library… is 10 times worse.

The long deprecation period would be super nice.

[llogiq]

We could effectively remove deprecated APIs for all libraries that do not specify their target version. Granted, the APIs would still be there, but they would be unavailable unless the library’s target version is e.g. 1_0.

[ArtemGr]

We could effectively remove deprecated APIs for all libraries that do not specify their target version.

That’ll make the libraries (and all depended crates!) break by default, wouldn’t it? I’d prefer a more durable defaults. For example cargo package could write down the current Rust version if it isn’t specified in the manifest.

If the author was able to package and publish the library crate then it should essentially “just work” as long as possible.