Proposal: Move some cargo config settings to `Cargo.toml`

tools and infrastructure cargo
1

This post proposes to replicate some .cargo/config.toml settings in Cargo.toml files. This way, we no longer need to use file system dependent config files for crate-specific settings.

Introduction

The behavior of Cargo can be configured through configuration files named .cargo/config or .cargo/config.toml. In contrast to Cargo.toml files, which specify properties of a Rust crate, these configuration files are only based on the file system hierarchy. This means that any .cargo/config file applies to all subdirectories of the current folder as well. If multiple config files are found in the hierarchy, their values are merged together.

The following things can be configured through configuration files:

The Problem

While there are some config keys that only affect the cargo executable itself, many keys also affect crate/package builds in some way. For example, many projects use the build.rustflags key to pass additional package-specific flags to rustc. This is problematic because config files are completely package-agnostic and only depend on the file system hierarchy and the current working directory. Thus, a build.rustflags key might not be applied (if the build is started from a different working directory) or accidentally change the build of a completely unrelated package (if the current working directory contains a config file). The hierarchical merging of config files makes this problem even worse since each config file applies to all subdirectories.

These problems were already reported many times. See the Problems of .cargo/config files and possible solutions thread on the internals forum for an overview.

Possible Solutions

One possible solution would be to change the behavior of config files, as proposed by josh. Instead of depending on the current working directory, Cargo would choose the config file from the package/workspace that is being compiled. Hierarchical merging would be disabled, with exception of the global configuration file in $HOME. The problem of this approach is that such a change is not backwards compatible and thus requires a migration period of some sort.

In this document, I want to view the problem from a different angle and propose an alternative solution: The problem isn't that cargo config files are package-agnostic in itself, but that projects are using package-agnostic config files for package-specific configuration. Instead of changing how cargo config files work, I think it makes more sense to move the package-specific configuration to the package related config file we already have: the Cargo.toml.

Proposal

My impression is that must people only use cargo config files because there is no better way to specify their settings. For example, passing additional package-specific arguments to rustc is not possible without a .cargo/config file. Setting a default target for a WASM crate is not possible without a .cargo/config file either. I'm sure that most people would prefer Cargo.toml options, if such options were available.

Not all .cargo/config settings are relevant in Cargo.toml files, of course. Some of them are clearly package-agnostic, so that .cargo/config files are the perfect place for them. Many others, however, would be useful to be available in Cargo.toml too. Below I evaluated all available .cargo/config options for this:

Option Might be package-specific, i.e. would make sense in Cargo.toml Why? PR
paths :x: Behavior already available through [patch] section, see Overriding Dependencies - The Cargo Book
alias (:white_check_mark:) doesn't affect the build, but allow shortening package-specific commands
build.jobs :x: should not affect build result
build.rustc (:x:) are there projects that require a custom rustc binary?
build.rustc-wrapper (:x:) are there projects that require a custom rustc wrapper?
build.rustdoc (:x:) are there projects that require a custom rustdoc binary?
build.target :white_check_mark: e.g. projects targeting a specific architecture such as WASM or embedded #9030
build.target-dir (:x:) maybe to incorporate a package with other build systems?
build.rustflags :white_check_mark: for package-specific flags
build.rustdocflags :white_check_mark: for package-specific flags
build.incremental (:x:) are there projects that don't work with incremental?
build.dep-info-basedir (:white_check_mark:) for third-party build systems?
build.pipelining :x: should not affect build result
cargo-new :x: package-agnostic
http :x: package-agnostic
install :x: package-agnostic
net :x: package-agnostic
profile.<name> :white_check_mark: clearly build-related; most of these settings are already available in Cargo.toml
registries/registry (:white_check_mark:) affects the dependencies of a crate?
source (:white_check_mark:) affects the dependencies of a crate?
target :white_check_mark: affects how a crate is built or run
term :x: only affects cargo output
unstable :x: most unstable options are either already available in Cargo.toml (e.g. metabuild) or package-agnostic (e.g. -Z no-index-update); there are a few exceptions, though:
unstable.build-std :white_check_mark: always required for packages built for custom targets #10308
unstable.build-std-features :white_check_mark: affects how a package is built
unstable.doctest-xcompile :white_check_mark: required for running doctest of packages built for custom targets
unstable.panic-abort-tests :white_check_mark: some tests might only work with this flag

(I'm happy to discuss and correct this classification!)

(Edit (2022-01-20): Updated the table with a new pull request column)

We see that only a subset of .cargo/config keys can be package-specific (the keys marked with :white_check_mark:). I propose to replicate all these config keys in Cargo.toml files, so that users can decide which file they use (package-dependent or package-agnostic). This way, we can solve most problems that people currently have with config files in a fully backwards compatible way.

The replicated config keys are not removed from .cargo/config files. Users can still use them to override the setting specified in Cargo.toml files, like it is already the case for the [profile] table.

Challenges

(I don't know much about cargo's internals, so please let me know if something is missing or wrong.)

  • The packages of a workspace might have different build.target fields after this change. Cargo needs to be able to handle this on cargo build --workspace, i.e. build the packages for different targets.
  • Similarly, the packages of a workspace might have different build.rustflags/build.rustdocflags, which might impose some challenges when sharing compilation artifacts across packages in a workspace.

Drawbacks

  • Since the old configuration options are still present in .cargo/config files, there are now two places for configuration, which could lead to confusion.
    • Clear documentation could mitigate the confusion
    • Over time we can deprecate any .cargo/config keys that are no longer needed

Alternatives

  • Change behavior .cargo/config files to disable hierarchical merging and look at package root instead of current working directory. This would be backwards-incompatible and introduce a lot of churn.
  • In addition to the proposed changes, remove the new Cargo.toml keys from .cargo/config files so that there is only one place to define things. However, this also has drawbacks:
    • Not backwards compatible
    • Users might want to set some keys in a package-agnostic way, e.g. turn on --verbose for all rustc invocations via build.rustflags
24 Likes
2

This could be problematic, from my understanding setting some flags such as -C target-feature can result in incompatible ABIs, so really need to be set for the build as a whole, not just per-crate. Maybe this could be restricted to only a whitelist of flags?

How would this deal with build.rustflags being set in both Cargo.toml and .cargo/config, would one outright override the other, or would the arrays be merged together somehow? For example I globally set build.rustflags = ["--cap-lints=warn"] to workaround harmful usage of #![deny(warnings)] in code, if I go into a project that sets local rustflags would I then override/be overridden by those completely just because of this one flag?

4 Likes
3

The -C flag can also pass arbitrary arguments to the linker so in theory it could do (almost) anything to the build as a whole, no? That said, there could be legitimate reasons for libraries to use certain linker arguments. But that can be very linker and situation specific.

4

More broadly it feels like the strong seperation between cargo.toml and .cargo\config.toml is often inadeqaute. At least for my Windows builds.

Rust tools out of the box don't support a lot of useful things for building Windows binaries. Manifests, setting the stack size, linking the vcruntime, etc. So I need to set custom options. Some of these I may want to change based on the profile (e.g. debug/release/test). I may even want to set these differently based on the binary being built.

I can hack round that by running cargo build in different directories for different kinds of builds, thus using different configs. Or by having adhoc scripts that set environment variables and command line options. Both of these options surprise people who expect to just use cargo build and have it do the right thing for all supported targets.

Though I don't know if addressing this is beyond the scope of this proposal.

EDIT: The tl;dr of this is it would be useful to have some "whole build" config that's more integrated into cargo. E.g. it's aware of profiles and the binaries being built.

3 Likes
5

I am completely in favor of this.

I think we should do both: replicate these settings into Cargo.toml, and stop searching upwards for additional cargo config files. The former will make the latter easier to work with.

(Note: I don't think "alias" should be replicated, as it is a matter of user preference rather than crate preference.)

1 Like
6

I'm in favor of this in general. But I suggest not moving rustflags as-is, because it's an inelegant solution that breaks Cargo's layer of abstraction and has a potential to break builds.

Instead, I suggest adding "Cargo-native" features for things that were previously hacked with rustflags. For example, Cargo should recognize a property like profiles.<target>.release.cpu and translate it into -C target-cpu flags automatically where necessary.

The same way there's profiles.dev.debug = true rather than rustflags = ["-g"].

24 Likes
7

Target specific settings is definitely something I’ve wanted for projects in the past, iirc it’s necessary for win32 cross-compilation linking.

2 Likes
8

I would really, really love something better for flipping on target-feature(s) for particular architectures, or at least enforcing that they're enabled in some programatic way.

I don't know what that looks like, but it'd be really nice to encode that information in some sort of structured way rather than putting it in the README/rustdoc and hoping people see it.

I'd almost prefer something to the effect of a compile error if you're on an i686/x86_64 target_arch and the target_cpu is less than sandybridge and/or the target_feature(s) for +aes,+sse2,+sse4.1,+ssse3 aren't enabled, to name a specific example:

I suppose that's possible today with lots of manual cfg gating and compile_error to spit out a semi-reasonable message. Sure would be nice if things would Just Work though.

9

The current behavior is that only one rustflags source is used (env variable or config value), they are not merged together. So the consistent behavior would be to make the rustflags value defined in .cargo/config override the value in Cargo.toml.

Given the potential problems with certain rustflags you mentioned, the variant proposed by @kornel below (adding "cargo-native" features instead of supporting rustflags) might be the better solution. This way, your example would continue to work.

1 Like
10

I would really like to see aliases for crates, as in larger crate workspaces I often have long strings for things like cargo run --bin crate_name -- <crate_args> that would be useful for other people who are working on the project and new contributors who want to do common tasks within the workspace.

3 Likes
11

Interesting.

My main concern would be conflicts with either users' own aliases or other Cargo commands the user has installed. I wouldn't want to see, for instance, a crate-specific cargo wasi alias that conflicts with the cargo-wasi command.

12

For what it's worth, the node world's npm run pulls arbitrary aliases from package.json .scripts, while the toplevel namespace is reserved by npm itself. (Though includes specific ones like npm start, npm test, &c which are aliases for npm run start, npm run test, and so on.)

Perhaps a cargo run --alias aliasname could cover the "long strings of things" use case?

13

Personally I think that there's good uses for aliases. The bootloader crate is phasing away from cargo-xbuild in favour of -z build-std and it allows for a more seamless transition. It also makes the build command way shorter, and if we need to modify the build flags the user doesn't need to know the new ones to compile the bootloader (eg if they wished to compile an older version they need that version's flags). I don't really like the --alias idea simply because "cargo build --alias cross" feels strange imo

14

We discussed this in the Cargo meeting today. We agree with this proposal and analysis, and we'd like to see the :white_check_mark: settings added to Cargo.toml.

(This includes aliases, with the caveat that we're still discussing the handling of aliases that would conflict with built-in commands.)

If people send PRs based on this thread, adding related groups of config settings (not all of them at once), including corresponding documentation updates to the Cargo manifest documentation, we'd be happy to merge them.

We look forward to seeing these added to Cargo.

17 Likes
15

For interested people, I have just sent Expose build.target .cargo/config setting as packages.target in Cargo.toml by Ekleog · Pull Request #9030 · rust-lang/cargo · GitHub that tries to add a package.target key, which hopefully will be a first step in implementing this.

1 Like
16

I've started thinking about porting build.rustflags to profiles.**.rustflags, and one question is bothering me: should this parameter also be available for crates on crates.io (as I've started going for the target flag), or should it only be available for local crates?

I overall think that it would be better to have it be available for crates on crates.io, but OTOH I wonder whether it could end up causing issues, if eg. crates start depending on nightly flags, or flags that cause arbitrary execution. (but there are already build.rs files… so it's hopefully not that big an issue) So I just wanted to raise the question, in case someone else would have a more informed opinion than I currently do.

PS: I'm assuming here that @josh's latest message was confirming porting rustflags to Cargo.toml, thus not leaning towards @kornel's solution (and I agree that it'd probably be best however cleaner @kornel's solution is, if only because one of the (major?) use cases of said rustflags is to use nightly rust flags and we probably don't want to add too much hassle for maintaining this on the rustc/cargo side)

17

As an update on this, I've made two tracking issues

3 Likes