Use tags on closed GitHub issues to communicate which supported and/or future stable versions are still affected.
Motivation
Under the release trains models it will take a usual (i.e. not backported) bugfix 2–3 months to reach a stable release. Closing the corresponding GitHub issue once the fix hits the nightlies can set incorrect expectations about its availability and makes it difficult to keep track of known problems in stable releases.
Detailed design
When an issue is found to be fixed in nightly and closed, tag it with the version numbers of the current stable and beta releases, signifying that despite it being closed it still is a known issue in these supported releases and the general availability of the fix is only expected in the subsequent release. If it’s fixed in beta, tag with the current stable release version only.
For instance, suppose that the release, beta and dev channels are at 1.2, 1.3, and 1.4 respectively. If a bug is fixed and its issue is closed today, this bug still is present in the current 1.2 release and absent a backport will be present in the 1.3 release. Therefore the issue is tagged with known-issue-1.2 and known-issue-1.3.
A library author looking at the issue will realize that they can’t reasonably assume the presence of the fix until Rust 1.4 is released. Additionally, anyone targeting e.g. Rust 1.3 can filter by the tag and see which bugs they still need to work around despite those issues being closed.
Drawbacks
It’s not fun to tag closed issues and easy to miss some.
Increased tags clutter?
Alternatives
Make this policy optional and apply to footguns and issues with big impact.
Tag the issue with the branch it’s fixed on (e.g. fixed-in-1.4).
Same as above but don’t close the issue until that branch becomes a stable release.
I think it’s better to indicate the version in which it has been fixed instead of the versions it’s affecting… Because when you add known-issue-1.2known-issue-1.3 it sounds like only those 2 versions are affected. But what about v1.1? And tagging every affected release gets very tedious.
Could this be automated? Once an issue is closed, the rust-highfive bot could add a fixversion-1.x label where x is just based on the closing date and the corresponding version. E.g., an issue closed today will be available in Rust 1.4 (the nightlies currently have version 1.4).
The only exception to this rule is stuff that will be backported.
I believe there are limits on the number of distinct labels you can have that the rust repo has already reached. We could have a bot that makes a comment with the fixed-in version, though.
Please start creating milestones again and when closing a genuine bug
issue (not WORKSFORME, WONTFIX etc. I don't think a bot could
distinguish them) assign the appropriate milestone.
Any issues automatically closed from the commit and PR messages would probably risk being left out.
I’m not sure exactly, but for example the Elasticsearch team stopped using milestones. My reading was that milestones need preplanning, while tagging for merge and release allows more for a “if it is done and flagged for release, put it in this bucket”.
Have a look at https://github.com/elastic/elasticsearch. They have tagged everything for the releases that the code attached to the issue was released in. Also, that allows for multiple tags, if there was a bugfix that made it into multiple production lines.
That doesn't matter for Rust, since it's 99% of "one milestone every 6 weeks".
That could be more important to us, if we wanted to show that something is not just fixed-in-1.4.0 but also fixed-in-1.4.0-beta.3. (I'm not sure I'd add labels for betas, though.)
I didn't mean that with preplanning. I don't have the impression that these milestones are planned in the sense of "this feature is going to be in this milestone", but more in the sense "if it reached stability and we trust it, we stage it for the next version in beta". So a milestone is never 90% finished (which Github Milestones expressed), but the project is "2 weeks from release point", which, as you say, can be solved with a calendar on the wall.