It needs a sym defined for the option name, and each of its values, and create a function listing all valid symbols.
It needs to define an internal enum or struct for its parsed value
It needs to define a parser converting from CLI string flag to the internal value
It needs error text added for the CLI parser, because the parser returns false instead of Err
It needs help text added to the list of CLI flags
It needs a cfg(option) defined
It needs UI tests for the new cfg(option) and each of its values
It needs to add the cfg to the list of disallowed user cfgs
It needs to add the cfg to the test of the list of disallowed cfg
It needs to bless the output of check-cfg tests and a few others
It needs to add the cfg to the list of allowed cfgs in CheckCfg
It needs check-cfg documentation updated
It needs a help chapter in the unstable book
It needs to be added to the list of unstable flags
It needs to be added to the default session configuration, gated on nightly options
It needs to be added to the test of options tracked by the session
It may need #![feature(option)] added to tests and libraries
Then in Cargo:
It needs to be added to util::toml structs for the appropriate Cargo.toml section
It usually needs a custom struct with a TryFrom or serde::Deserialize implementation to parse the value from multiple TOML representations.
It needs to be added to the list of nightly Cargo features
It needs help text and parser added for the nightly Cargo feature flags
It needs nightly-only tests checking the TOML parsing and the -Z flag
It may need additional logic and tests for passing the option down to rustdoc
It needs to be added to internal structs representing options
It needs to be added to the hash tracking build options
It needs to be added to code serializing rustc flags, and update countless Cargo text output tests that expose the flags
It needs to be added to functions constructing default internal options and functions converting from TOML-deserialized structs
It may need to be added to config.toml representation, and have code added for merging of options, along with tests for parsing TOML as well as test automagic mapping from env vars to config.toml options
It may need to be exposed as an env var in build scripts, along with bespoke tests for this
It needs to be added to Cargo's option reference.
and then when stabilizing:
The nightly-only option gates need to be deleted.
The lists of unstable options need to be updated to make them stable, separately for rustc and Cargo.
Tests need to be updated to test the stable version of the CLI flags.
Cargo output tests need to be updated to allow the stable flag.
That's A LOT, and this doesn't include any of the actual functionality behind the flag! These steps are just for making a flag exist, and get passed between Cargo and rustc.
A few steps like definition of the flag, its syntax, and its documentation are obviously necessary. But a lot of steps around parsing, serializing, and testing of the parsers and serializers seems to needlessly require bespoke code and bespoke tests.
Do all compiler options need to be mapped to cargo though? I saw some recent discussion about adding rustflags support to cargo (just for bin-crates I think), which would make part of this pass-through.
Not all flags, there certainly are some low-level plumbing ones that aren't useful for Cargo or don't map 1:1 to something in a manifest.
However, I think it's common that if some new functionality or configurability is added to Rust in rustc, that Cargo users would also want to be able to use it (e.g. fmt-debug and location-detail should be in Cargo profiles). For majority of Rust users Cargo is the primary way they build the code, so Cargo manifests and config are the right place to set the options.
The current RUSTFLAGS env var is a pretty bad interface. You can set only one list of flags for all invocations of the compiler, for both build-time and final binaries, and for both the host and (cross) compilation target. Any change to flags invalidates the build cache, even when the flags have a limited effect. This is coarse and fragile.
I think existence of profile.rustflags is a symptom of the problem I'm highlighting here. It's just so hard and tedious to add individual options to Cargo.toml sections like [profile], that it's easier to give up on TOML, give up on using structured data types, give up on consistency with the rest of the manifest, give up on smart and selective merging of options, and just shove every existing and future option into a single opaque list of strings, using a flat CLI syntax instead of TOML, inside TOML.
RUSTFLAGS are opaque to Cargo which makes it hard to reason about. For this reason, the general feel I have for the Cargo team is we view them as an escape hatch for special cases or when we've not yet come up with what abstraction belongs in Cargo. We even have Tracking Issue: Exposing RUSTFLAGS in cargo · Issue #12739 · rust-lang/cargo · GitHub for finding the appropriate abstraction in Cargo for RUSTFLAGS people use. Note this doesn't mean we'll always do a 1:1 between rustc and cargo.
I believe its 1.85 that will at least stop thrashing the cache on RUSTFLAGS changes but will instead keep a distinct cache per RUSTFLAGS.
If the flags do have a limited effect, then that is an example of one of the considerations Cargo needs to take into account when abstracting RUSTFLAGS. For instance, the flag might only be relevant for linking of the final binary. LTO used to be in this category but we then sped up the builds of all crates by building less when LTO is activated.
Couldn't a lot of this either be automated or code generated? E.g. Have a central toml file of all flags and whether they are unstable, allowed as cfg, help text etc. Then everything else can be code generated from that using build.rs or proc macros.
I assume you then need a separate such file for cargo, but perhaps it could even be a shared file.
This is how we have solved similar problems at work. If adding a new configuration needs updates in 10 places of just boilerplate: Automate it!
I left something implied in this that should probably be made explicit: Cargo should not blindly expose rustc flags as profile settings. Instead, consideration is needed for how the feature affects Cargo users, including
How it should be fingerprinted
Whether it is a platform-specific setting or not and if its being exposed in a place that affects all platforms
The right level of abstraction
These are decisions that the Cargo team needs to make. If we blindly code-gen off of data from rustc, then we are likely to miss that.
Related to having a shared source of truth, a problem we haven't worked out yet is how to deal with the release side of this. The rust repo would depend on the cargo repo which would depend on the rust repo. However we share this source of truth, we need to make sure release processes are updated to handle it and need to avoid losing updates as we ping-pong code across teams/repos. The other context this is coming up is in discussing solutions around [lints], especially Cargo lints.
I absolutely agree the profile options should still be exposed on case by case basis, and even have an option to use a different format where necessary.
Still, common definitions, parsers and serializers would save a lot of time and boilerplate compared to the status quo. I imagine compiler options defined on a level of clap/structopt, while the current code feels like using bare argv.
Creation of a circular dependency between rustc and cargo is a problem indeed. It could work if the shared options crate lived in another repo, but submodules are a pain. Having them in a crates.io crate would be weird as it'd require being published mid release cycle. Currently rustc could read definitions from Cargo, but that's a wrong place to put rustc flag definitions:/
Maybe rustc --print options-as-json that a Cargo helper crate could run during build?