The Life and Death of an API

[Gankro]

Now that the trains are running, and subteams have been established as a thing, we can make some desperately need changes to The Process of making changes to APIs.

I posit the following premises:

• The amount of concensus building and process a change requires should be proportional to controversy, importance, and size of a change. Hard things should be hard; Easy things should be easy.
• RFCs are heavyweight
• PRs are lightweight
• RFCs are only useful if a significant and diverse portion of the community actively consumes and discusses them
• The more RFCs there are – particularly trivial ones – the less likely people are to interact with them. This may be due to exhaustion, being overwhelmed, being too busy, or being unable to effectively identify the RFCs that concern them. This is particularly important because professionals and domain experts are exactly the people we want chiming in, but also have the least time. We are surely lost if the only people who can deal with RFCs are hobbyists with all the time in the world.

As a result, I propose the following Life and Death of an API:

• Someone wants to make a change:
  • The change is sweeping, or an entire new module/crate/major-API: Submit an RFC
  • The change is to something unstable, or just adding a method: Submit a PR
  • Not sure: Submit a PR (particularly more friendly for new contributors, or those that are uncomfortable with RFCs)
• Someone reviews a PR, after normal discussion:
  • If it’s clearly not acceptable: Close the PR
  • If it’s clearly useful or filling a hole: Merge as Unstable
  • If it’s not clear: Ping the larger team for input
    • If concensus still can’t be reached: Block on an RFC
• If a change makes it through the whole RFC process: Merge as Unstable
• An unstable API has been around for at least one full release cycle, the team should discuss it:
  • If demand is high for stabilization, or the API otherwise seems to be pulling its weight: Mark as stable in the next beta
  • If an API isn’t pulling its weight: deprecate for removal in next release (giving an opportunity for community objections)
  • regardless of the action taken here: announce this transition in weekly team notes (maybe also some release notes?). The community should be aware of this, and able to provide feedback!
• Deprecations of stable APIs requires an RFC
• Due to the way that trait implementations effectively bypass the Trains, they may always require an RFC, except for perhaps the most trivial cases (like missing Debug or Clone impls)

The main point is: everyone should favour PRs, with RFCs reached for only for major changes or for achieving concensus where the team can’t. However the community should be able to interact with and track the process at all levels. The team’s responsibility should be on keeping changes flowing freely and seek broader community input only when necessary/ready. The larger community shouldn’t be bothered with day-to-day development.

[sfackler]

It seems like there may be somewhat of a circular dependency problem with certain kinds of unstable features - they aren’t compelling enough for people to jump off the stable train to use them, but we don’t want to stabilize them until people have had a chance to try them out and make sure they actually fulfill their purpose properly. One way to get more eyes on a feature when we’re considering stabilizing it is to have a “final comment period” for the feature, like this one for the Debug builders: Final comment period for Debug Builders stabilization. At the very least, it’s a “speak now or forever hold your peace” message to get people to read over the API and be aware of its existence. We’ll have to be careful to keep them from turning into huge bikesheds though.

[pnkfelix]

Hmm… a big reason we added the RFC process was that people were getting upset that their PR’s were being rejected (especially when the PR represented a significant amount of effort to develop). The reasons for rejection I’m thinking of were along the lines of “does not fit with long term plans for the language/libraries”

Your proposal makes sense to me for small PR’s where the construction effort was small and impact on existing code is also small. But it seems like there is likely to be a threshold on PR size where the policy outlined here will put us back where we started.

So maybe this should be at least acknowledged in the policy, that some broad or fundamental changes can/should jump straight to an RFC?

Update: I guess the proposal did have an item for jumping straight to RFC… maybe the text is fine, I don’t know, I still worry about backward steps…

[td_]

Deprecations of stable APIs requires an RFC

What is the course of action deprecation entails? Removing a stable API would require a major version number change, so would it be considered non-exceptional to deprecate a method indefinitely, issuing a warning whenever it is used from now on? If not, deprecating a stable API would be a bigger issue than an RFC I would think.

[glaebhoerl]

The more RFCs there are – particularly trivial ones – the less likely people are to interact with them. This may be due to exhaustion, being overwhelmed, being too busy, or being unable to effectively identify the RFCs that concern them.

Your post is about API changes, but this is also a very real phenomenon for language changes (which I tend to be more interested in) - except there it’s particularly the nontrivial ones. When the team declares some area, feature, or capability to be an urgent top priority, and there’s also significant diversity of opinion about how it should be accomplished, it tends to result in a cavalcade of long and detailed RFCs, and even longer discussion threads, which can be nigh-impossible to keep up with. My github notifications list still looks like this today:

(and I think there had been one or two more which I accidentally clicked on without reading)

I’m already dreading the possibility of this machine spinning back up for memory allocators / garbage collection and the Servo inheritance-ish grab-bag wishlist. I’m very interested in these topics, but can only take in so much material at a time - much more than that, and things take a turn onto the forever procrastination doom-loop (a bit like trying to allocate a too-large block from a fragmented heap). Doing it this way was perhaps unavoidable for certain topics which had to be got out of the way for 1.0, but now that we have that behind us, could we maybe think about taking the remaining big-ticket items in a more deliberate, “it’s done when it’s done” pace?

[Gankro]

What is the course of action deprecation entails? Removing a stable API would require a major version number change, so would it be considered non-exceptional to deprecate a method indefinitely, issuing a warning whenever it is used from now on? If not, deprecating a stable API would be a bigger issue than an RFC I would think.

While it’s not a legit breaking change to deprecate something stable, it is disruptive. Ideally, no one uses any deprecated APIs, so we’re still pushing them to change their code in a major way. It also effectively contradicts the pre-existing consensus that stabilized the API. We have to make sure there’s an appropriate migration path and that no one’s usecase is unfairly ignored.

Your proposal makes sense to me for small PR’s where the construction effort was small and impact on existing code is also small. But it seems like there is likely to be a threshold on PR size where the policy outlined here will put us back where we started.

Yes, I think a decent heuristic might be “Which is more work? The code or the RFC? – do the easier one.”

Regardless I think we can reserve the right to basically punt a PR to an RFC or vice-versa if we think the scale of the change merits it.

Your post is about API changes, but this is also a very real phenomenon for language changes (which I tend to be more interested in) - except there it’s particularly the nontrivial ones. When the team declares some area, feature, or capability to be an urgent top priority, and there’s also significant diversity of opinion about how it should be accomplished, it tends to result in a cavalcade of long and detailed RFCs, and even longer discussion threads, which can be nigh-impossible to keep up with.

I am hopeful that this is indeed a thing of the past; there are no more hard deadlines (missing a train is fairly trivial, except for maybe emergency soundness fixes). The “final comment period” system can also serve as a reasonable filter for you if you’re overwhelmed (hopefully).

It seems like there may be somewhat of a circular dependency problem with certain kinds of unstable features - they aren’t compelling enough for people to jump off the stable train to use them, but we don’t want to stabilize them until people have had a chance to try them out and make sure they actually fulfill their purpose properly. One way to get more eyes on a feature when we’re considering stabilizing it is to have a “final comment period” for the feature, like this one for the Debug builders: Final comment period for Debug Builders stabilization. At the very least, it’s a “speak now or forever hold your peace” message to get people to read over the API and be aware of its existence. We’ll have to be careful to keep them from turning into huge bikesheds though.

Great point! Although I kind’ve just pictured mark-as-stable-in-beta as the “final comment period” (a comfy 6 weeks!). Basically changes move unconditionally forward on the trains unless significant dissent occurs. There is perhaps danger of small changes getting rubber-stamped, though.

[eridius]

I think there is a bit of a middle ground here, which is an issue filed on rust-lang/rfcs (as opposed to a PR). These “RFC issues” can be used to gain a bit of consensus (or test the waters before spending the time implementing something) without going through the full-blown RFC process.

[td_]

While it’s not a legit breaking change to deprecate something stable, it is disruptive. Ideally, no one uses any deprecated APIs, so we’re still pushing them to change their code in a major way. It also effectively contradicts the pre-existing consensus that stabilized the API. We have to make sure there’s an appropriate migration path and that no one’s usecase is unfairly ignored.

I think you misunderstood me but you may have answered my question regardless? I meant: is deprecating stable APIs (and then keeping them around, deprecated, until Rust 2.0) a tool that the team has available to use to shape how std is used? Your response implies yes.

I had just thought before that deprecating was just for warning that the API will soon be removed.

[Gankro]

I’m not sure how the two options you’re stating are distinct, really. The only way you can really shape how an API is used is by adding or removing elements from it. (What are docs? ;))

[sfackler]

Deprecation has historically meant that it would soon be removed in the pre-1.0 world, but that will no longer be the case in the future since we don’t want to break backwards compatibility all over the place. There are certainly APIs that we will be deprecating, some even in the near future like Condvar::wait_timeout_ms.

[td_]

Yes but you can’t remove elements from a stable API without incrementing the major version number. So I had thought that deprecation would not be used on stable parts of std except as a part of the preparations for a shift to Rust 2.0. I just didn’t know APIs could be deprecated without a timetable for removing them. It doesn’t matter either way & this is off-topic, I was just asking for clarification.

[eridius]

Without ever having really researched the topic to see if this has been addressed, my assumption has been that API can be deprecated without having a specific timeline for removal, and that deprecated APIs would most likely be removed on the next major version number change (unless it was deprecated recently enough that it’s felt that it hasn’t had time for the community to raise any issues about it).

[lambda]

There are already things that are marked as deprecated between 1.0 and 1.1. I think that deprecation will be used any time that it’s determined that there is a substantially superior alternative. It’s true that those deprecated APIs can’t be removed until 2.0, but it’s not the case that APIs will only be deprecated as part of a preparation for 2.0, just any time there’s a better alternative.

[td_]

Thanks for clarifying

[aturon]

Thanks for getting the ball rolling on this, @Gankro, and sorry for the late reply. I think the proposal here is in the right ballpark, but I have some concerns – play by play below.

The amount of concensus building and process a change requires should be proportional to controversy, importance, and size of a change.

That makes sense, but experience has shown that the amount of “controversy” (which is to say, people having opinions and wanting them to be heard) can be very difficult to predict in advance. To some degree you just get a better sense for this over time, but it’s still easy to make mistakes or get surprised.

That said, we plan to have final comment periods for both RFCs and for ungating features. The latter means that even if a feature comes in directly as a PR, there will still be a chance for people to try it and comment on it after the fact, before it is fully committed.

RFCs are heavyweight

Part of the weighty feeling of RFCs in the past has been the lack of things like subteams and a visible RFC pipeline to keep things moving. But it’s also very important that “straightforward” API changes be easy to land with a minimum of process overhead, and I think eliminating the RFC requirement where we can is a good move.

The change is to something unstable

Hm, this is a bit worrying, at least as stated. Major revamps of unstable APIs should probably require at least amendment RFCs, but I’m not 100% sure where to draw the line there. Certainly in the past I’ve done some significant revision between RFC and implementation…

just adding a method: Submit a PR

I think I’d broaden that a bit to adding API surface that clearly follows existing conventions (no serious new design questions) and addresses a fairly clear need.

One worry here is bloat in the standard library, which can hurt discoverability etc. Personally, my feeling tends to be that if a new API fits in very naturally to the existing surface and follows conventions, there’s relatively little “cognitive overhead” and I’m happy to see it land, but I know @alexcrichton for example is more conservative on this front.

I think figuring out the exact line here is probably the most important – but also the hardest – piece of this puzzle.

Someone reviews a PR, after normal discussion:

These bullets make perfect sense, assuming we have fairly clear guidance on the previous question.

In particular, I want to draw attention to a new step here that the subteams make possible: the ability to reach for consensus at a “medium” scale – the subteam level – without going through the full RFC process. Assuming the subteam is active and fairly representative of the broader community, this should be a good way to keep things moving quickly while getting reasonably diverse input.

An unstable API has been around for at least one full release cycle, the team should discuss it:

I very much like the steps you outline here, but I’m a little unclear on the timing. In particular, as @sfackler talks about, it may be hard to get a good enough sense of “real world” usage to know that an API is fully baked. The final comment period does help with this, but I wonder basically which side we should err on: better to have more APIs in unstable limbo (and wait for a chorus of requests to stabilize), or to commit to more APIs that we are not fully sure about (but no one seems to object to)?

I don’t know what the right answer here is. One of the things I’m working on right now is pulling together all the info we need to have a libraries dashboard, and that at least should give some better visibility into the “unstable limbo” features.

The main point is: everyone should favour PRs, with RFCs reached for only for major changes or for achieving concensus where the team can’t. However the community should be able to interact with and track the process at all levels. The team’s responsibility should be on keeping changes flowing freely and seek broader community input only when necessary/ready. The larger community shouldn’t be bothered with day-to-day development.

This sounds very good, assuming that we can maintain a broad level of trust in the subteams (which is important anyway). Thanks for writing it up!

Yes, I think a decent heuristic might be “Which is more work? The code or the RFC? – do the easier one.”

+1

[bluss]

I agree with you Gankro, but I think:

• Some changes have much bigger impact than the diff tells, so this must be weighed in too. An example is std::thread::catch_panic that needs a big discussion for a small function.

• We must not make it too easy to add to the libstd. Keep quality high and don’t cave into providing just one more method.

[aturon]

Curious what the other library subteam members think about this overall? @alexcrichton @sfackler @Kimundi @BurntSushi @huon @brson

[Gankro]

That makes sense, but experience has shown that the amount of “controversy” (which is to say, people having opinions and wanting them to be heard) can be very difficult to predict in advance. To some degree you just get a better sense for this over time, but it’s still easy to make mistakes or get surprised.

Just to clarify, the point on controversy is a reactionary one. That is, when you submit a PR and no one can agree on it, this warrants an RFC. I suppose you can “anticipate controversy” and short-circuit to an RFC, though.

Regardless I broadly agree with your points, with the note that I am personally very willing to let us make horrible mistakes all the time. The trains should, ideally, guard us from those mistakes making it to stable, though.

[aturon]

Just to clarify, the point on controversy is a reactionary one. That is, when you submit a PR and no one can agree on it, this warrants an RFC. I suppose you can “anticipate controversy” and short-circuit to an RFC, though.

Yeah, that seems fine. I worry more about the case that the PR lands and then outrage ensues, but:

Regardless I broadly agree with your points, with the note that I am personally very willing to let us make horrible mistakes all the time. The trains should, ideally, guard us from those mistakes making it to stable, though.

It may be that I’m coming too much from a pre-1.0 perspective, where we had to move fairly quickly on stabilization decisions. Maybe the concern is moot now.

Anyway, main point is: controversy is not always easy to anticipate, and sometimes only crops up after the fact.

[alexcrichton]

Overall I really like what you’ve put together here, I like the rule of thumb of “do what’s easiest to start out with” as it’s very newcomer friendly and we can always recommend a different course of action over time.

If it’s clearly useful or filling a hole: Merge as Unstable

@aturon predicted my response to this well, but I’m personally hesitant to merge new features that are “just unstable” because we’re able to remove them later. Having a huge amount of unstable content in the standard library will greatly affect the experience when browsing the documentation, and I do fear that despite the trains there may still be a worry of defacto stability (e.g. if everyone’s using nightly b/c it’s got all the cool methods).

Another thing I worry about is that we have historically been quite reluctant to remove basically anything, so while although in theory we can remove an unstable API at any time (even if we very clearly say we will after a full cycle), we may not always be able to.

I would prefer that we only merge PRs which are on a clear path-to stable. My personal threshold for this is “I’m comfortable with this being insta-stable” as opposed to “let’s see if demand is high enough later”.

If a change makes it through the whole RFC process: Merge as Unstable

I do also want to emphasize here though that just because something makes it through the RFC process does not mean that we should merge it. Many RFCs often uncover interesting details which were not addressed during an implementation, and this can sometimes uncover a fatal flaw which may block landing the implementation altogether. This is very situation-dependent, of course.

I still worry about backward steps…

Yeah I definitely agree that we don’t want to swing the pendulum in the direction of “PR everything” like it was before. I do also agree, however, that right now the pendulum is a bit too far on the “RFC everything” side of things and we need to work on striking a better balance between the two extremes.

Although I kind’ve just pictured mark-as-stable-in-beta as the “final comment period” (a comfy 6 weeks!). Basically changes move unconditionally forward on the trains unless significant dissent occurs.

I like the idea of an explicit final comment period as it makes the feature highly visible (it’s easy to miss a PR going through), but this is also connected to my previous worry about how we rarely remove things. I also would prefer to avoid lack-of-communication as consent. An example of this is that although the debug builders had been unstable for quite some time, we didn’t feel that they had enough visibility/usage, hence the internals post (which worked out quite well!).

---

Another overall point (which @aturon mentioned as well) is that I think it’ll be super important for us to get some form of dashboard for all the unstable APIs we have. I think we have an alarmingly large number of unstable APIs today under std_misc and core still, and I’ve just been losing track over time what’s there and what their status should be.

Being able to get a bird’s eye view of where your favorite feature is and being able to ask the team to consider stabilization (or submitting a stabilization yourself) will be super helpful in making sure that nothing falls by the wayside. I’d also just love to see what features have been unstable for the longest (e.g. what should be actively decided on or removed).