cargo clippy -- --deny=warnings will catch any compiler warnings, but it doesn't catch cargo warnings, such as warnings about unknown Cargo.toml fields or user-defined ones as created by println!("cargo:warning=this-is-a-warning").
I wonder if we can find a natural way to make one of the cargo commands fail if there is such a warning.
Probably this want is somewhat niche; in our case we emit custom warnings if some optional build tools are not installed, but CI should always have all of them.
cargo clippy --help output is not too long, if we add a new option we can put it there.
I think its reasonable for us to consider how to allow doing this. We also have many other types of warnings in Cargo and are developing a linting system for cargo.
RUSTFLAGS wouldn't quite be the appropriate mechanism for controlling this though. That is a low level escape hatch for users to do things that Cargo doesn't currently support and there are compatibility and other hazards with it. Instead, we should be looking for how people use RUSTFLAGS and Cargo-native ways of exposing those features (Tracking Issue: Exposing RUSTFLAGS in cargo · Issue #12739 · rust-lang/cargo · GitHub).
Specifically, a Cargo-native way to deny warnings has been prototyped as CARGO_BUILD_WARNINGS=deny. So far, this only controls warnings from rustc and not any of the warnings from Cargo but we leave the door open for expanding that further. In the tracking issue, we even call out build script warnings.
Something that may be helpful here is to understand why you are emitting build script warnings. In addition to a general better understanding of use cases in considering features, I'd like to understand how this will play out with "cap lints". Generally, warnings are suppressed for transitive dependencies and -Dwarnings won't break your build for a warning in a transitive dependency. Build scripts are an interesting case because one role they play is to generate code that is adapted to the machine they are running on. This could make a warning from a transitive dependency very relevant.
we're using msgfmt (for translations) and sphinx-build (for generating man pages),
the results of both are embedded into the binary. If these tools are not available, everything else still works fine except for translations and docs. Of course tests can catch that too, so it's not super important.
So sounds like this is pure-input/output codegen that you are performing directly, and not something others will run. I would be interested to hear from others using warnings to see if there is something more we could learn.
by the way. have you considered snapshot testing for your codegen? The idea is that instead of using build.rs to generate content if changed, you move the generator to a test and commit the result. The test will fail if they diverge (to catch direct edits) and you can use an environment variable to "bless" the new generated output, overwriting whats committed instead of erroring.
When used in a library, this can be a big help to build times for dependents because the code generator and its dependencies are no longer built since callers don't care about dev-dependencies / tests.
For applications, this could likely still offer build speed improvements because the code generator is no longer in the critical path but is now being built with the tests, offering more parallelism, and being run in parallel to other tests.
The main trade off with this approach is if the code gen source has a high velocity of changes. In your case in particular, there really isn't a warning mechanism for tests or a good way to dynamically ignore tests and report a cause. I am working on both but that is unfortunately one of my lower priorities atm.
Cargo does not display any cargo::warning if the script succeeds. I think this makes it different from language warnings that are displayed regardless whether compilation succeeds or not.
I've been relying on this hiding behavior, and using cargo::warning like log::debug (because the actual log output from build scripts is a chaotic cacophony, and I need to bring attention to one or two important lines in the log, among the hundreds or even thousands lines that cargo dumps uncritically).
If my build script encounters a problem that is possible to recover from, then I emit a warning immediately and proceed with fallback. If the fallback succeeds, then it's fine, and I rely on the warnings being no-ops [1]. Only when the fallback fails, user is automatically notified. And then it's desirable that it's not only error about the fallback, but also the earlier failure of the first attempt.
For example in a sys crate, if pre-installed library is not found on the system, that's a cargo::warning, but it's recoverable by building a vendored version from source. User may want to use the vendored version anyway, so turning warnings into errors would break the script.
I can't make my build scripts not rely on fallbacks and require an explicit choice, because Cargo doesn't have global or mutually exclusive features, and in Cargo it's unreasonably difficult to disable features of transitive dependencies, so many strategies/fallbacks will inevitably be enabled at the same time, and it has to work.
except when users enable verbose output, but then they may be trying to debug build issues, so log::debug-like usage is fitting ↩︎
by the way. have you considered snapshot testing for your codegen?
For translations, there are two phases
we compile with a special --feature that writes every gettext!("some message") found in Rust sources to gettext's *.po files.
(on every expansion of this macro, we open a file, write to it, and close it.. instead of a special --feature, we have considered having cargo test drive this (like the ts-rs project does) but IIRC that would have been worse).
That part is snapshot-tested; we track the *.po files in Git. (Whereas the cargo build-man check uses git status, we have a pure test that checks for staleness of the Git-tracked *.po files without overwriting them. This is nice when running the test locally as opposed to in CI).
in ordinary builds, we use build.rs to call msgfmt to transform *.po files to a format that's easy to parse,
parse that and convert it to a Rust file in $OUT_DIR containing a static phf::Map with all translations
Then we include that generated file with include!(concat!(env!("OUT_DIR"), "/localization_maps.rs")); in lib.rs.
We could add the localization_maps.rs codegen result to Git,
but we'd need to measure any build speed improvements.
It might improve the critcial path but at least caching already works well here; this only needs to be rebuilt whenever *.po files change.
In future, we might switch to Fluent for translations,
which does not need phase 1 at all, and no external subprocesses in phase 2 -- I imagine we'd parse *.ftl files either at compile-time or runtime but no need to add to Git.
For man pages, our requirements should be more or less similar to cargo's except that the exact output still depends on the version of sphinx-build used..