Split the current Tier 2 level of platform support into a new Tier 2 and Tier 3, moving the current Tier 3 down to Tier 4. The new Tier 3 would have the same definition as the current Tier 2. The new Tier 2 would have similar requirements as are in place for officially supported tools; it would require that CI be present for that platform and running tests, and releases are gated on CI passing or the platform being demoted to Tier 3, but not every single commit is gated.
At the moment, Rust has Tier 1 level support for three operating systems, Window, macOS, and Linux, and two ISAs, the 32 and 64 bit x86 architectures; a total of 7 targets, with two different Windows toolchains (GNU and MSVC) and no 32-bit macOS build.
Tier 1 status in the Rust project means fully supported; there are builds of both the compiler and standard library available, and the Rust CI system runs a full build and test for the platform on every commit before it is merged to master.
Rust has Tier 2 support for 77 different targets, with 36 different ISAs, ~13 different kernels (including bare metal), and with several different toolchains (GNU, Android, MUSL, etc) for some of the kernels.
However, Tier 2 support has a much broader definition; it just means that some subset of the Rust project is guaranteed to compile for that platform, by having that a mandatory part of CI. That subset could be just core, std, or the full compiler. And it makes no guarantees about tests.
Tier 3 is then the catch-all for other ports present in the tree; there may be some code to support those platforms in Rust source, but there is no official CI setup for them and no guarantees made by the Rust project. This may be because the toolchain is proprietary, the platform is niche, or just no one has taken on the work to set up CI for them.
Tier 1 and Tier 3 seem to have fairly reasonable definitions, but Tier 2 can indicate a very widely varying level of support in practice. While building on a platform in Rust gives a greater sense of certainty than building in, say, C or C++, it's still no substitute for having the full test suite run. Downstream distributors have noticed this; the Debian packaging of Rust disables the test suite because (among other reasons) it builds and runs on a wider variety of ISAs, and the Debian maintainer couldn't keep up with getting the test suite to run reliably on platforms not supported by the core project; and additionally, Debian imposes self-hosting requirements on many platforms, while the Rust project Tier 2 definition only requires cross-compiling.
Additionally, since Rust 1.0, I don't believe that any target other than the -msvc targets have migrated to Tier 1 status. At the moment, there are RFCs for defining the target tier policy more formally, to write down the unwritten rules for when a target can be promoted or demoted, as well as one for promoting aarch64-unknown-linux-gnu to Tier 1.
However, there is a fairly big gulf between the bare minimum definition to be included in Tier 2 (there must be at least a working cross-compile of libcore to the target platform), and what is required to promote to Tier 1 (all of libstd working and testing).
I would like to propose breaking Tier 2 up up into Tier 2 into Tiers 2 and 3. The new Tier 3 would have the same definition as the current Tier 2, and the new Tier 4 would be the same as Tier 3. The proposal would add a new Tier 2 which has stronger requirements than the current Tier 2, but weaker requirements than Tier 1.
The new Tier 2 under my proposal would have the following definition:
- CI will build the full compiler (if applicable) and standard library for the platform
- CI will guarantee that the build succeeds before merging to master, just like the current Tier 2 definition
- CI will additionally do a self-hosted build (if the target is a and run the test suite on the platform, but failures will not block merging to master. Failures will be noted in a similar way too toolstate, to make it easier for users of that platform to avoid updating to broken versions.
- Releases are guaranteed to include working builds for this platform; during the beta period, regressions introduced to the test status will be fixed, marked to ignore if not relevant, or the platform will be demoted to the new Tier 3 status (equivalent to the current Tier 2 status)
This new Tier 2 solves a couple of problems. It provides a better stepping stone towards Tier 1, as you can evaluate more easily how prompt the community is at fixing regressions introduced, including regressions affecting the test suite. It allows for working out the kinks in a platform's CI before committing to Tier 1 support. It also gives users and downstream distributors a better sense for how well supported a platform really is, as "does core build" is really a fairly low bar for a platform to pass.
Note that the new Tier 2 is not intended only as a stepping stone to Tier 1; some platforms may stay in Tier 2 indefinitely, if they are more niche, have slow or flaky build systems, or have other reasons why gating every build on tests passing would be excessively burdensome.
One possible alternative would to keep the three tiers, but add information to Tier 2 on self-hosting status and testing status; for instance, adding an extra column about testing and self-hosting status in CI. I think that there are a few reasons not to prefer this; for one, Tier 2 already has a lot of different platforms with varying levels of support (core, std, and the full compiler). For another, whether the test suite is passing and whether the compiler is self-hosted are different axes than which components are present, and what level of commitment there is to that status, so it makes some sense to present that information in a different way.
Another would be to add sub-tiers to Tier 2, such as Tier 2A, Tier 2B, etc, depending on different levels of support. That seems to be somewhat more confusing than just adding tiers. Adding a new tier only causes some confusion during the transition, which can be helped with good documentation, while sub-tiers add additional cognitive load as long as they exist. Additionally, I don't know that anyone relies on tiers in a machine readable way, it's just a way of documenting both internally and externally the level of support expected for a given platform, so a transition to an updated numbering shouldn't be too difficult.
Another alternative would be to use this definition for Tier 2, and just demote everything which doesn't meet this definition to Tier 3. That would mean that Tier 3 might have a mix of platforms that are built and available via rustup, and ones which are not, but Tier 2 would now only have platforms which are guaranteed to work as they are passing the test suite and (if applicable) self-hosting.
A third alternative would be to use this definition to replace the definition of Tier 1. At the moment, there's a fairly high bar to enter Tier 1, because having every change gated by the platform compiling and passing the full test suite is a high bar, and any issues with flaky CI hosts, flaky tests on certain platforms, slow build and test times, or difficulties in contributors being able to get access to the platform to debug and test fixes, means that it's worthwhile being very conservative before committing to Tier 1. However, if the Tier 1 definition were loosened a bit, along with a subset of platforms (such as the current Tier 1) being designated as dev platforms which are guaranteed to have every build pass the test suite, would allow for downstream users of stable releases to trust that the platform has been stabilized during the beta period, while not introducing too much additional burden on development.
The original pre-RFC suggested that the new Tier 2 definition should include that the platform is a host platform with a self-hosted compiler, but it was pointed out that not even the current Tier 1 definition includes that requirement. The proposal has been updated so that hosting is now orthogonal, so the new proposal is just about self-hosting if applicable, and running the test suite.
Does this, or something similar, sound like reasonable fodder for an RFC? If so, I can edit the above into a more RFC-like format and submit it.