Pre-RFC: Presets for Cargo

tools and infrastructure cargo
1

RFC repo: GitHub - crumblingstatue/cargo-presets: An (upcoming) RFC about presets, and an implementation using a cargo wrapper

This RFC has largely been controversial, so I'm considering abandoning it in favor of an argfile type solution. Thank you @pitaj for the suggestion!

EDIT: It's not viable, see https://github.com/rust-lang/cargo/issues/13690#issuecomment-2032508241

Inlined:

Summary

Add presets to cargo, which are user defined build configurations stored in a file, with the intent of making it easy to switch between build configurations, and keep tooling in sync with cargo.

Motivation

Projects often have different build configurations for different purposes. For example an application might have different backends, or it might be buildable for the web using WebAssembly.

Currently, if someone wants to build such a specific build configuration, they always have to provide the right flags to cargo for every invocation.

Here is an example of what those flags might look like:

  • Web build configuration
    • --target=wasm32-unknown-unknown
    • --default-features=false (there might be features that need to be disabled on the web)
    • --features=web,specific,features
  • Configuration for an alternate backend
    • --default-features=false (disable the default backend)
    • --features=alternate-backend

These flags also have to be set independently for every piece of tooling, most notably Rust-analyzer. Rust-analyzer has its own configuration options for target and features, which the user has to set independently of other tooling, or manual invocations of cargo.

There are other examples of commonly used tooling, like Bacon, which also require passing the right cargo flags to.

Conflicting cargo flags arising from forgetting to pass the exact right arguments for every tooling and cargo itself will invalidate the current build, and can be the source of a lot of unnecessary lengthy rebuilds.

If there was a way to define a set of build configurations (presets), the user would only have to remember the name of the preset, instead of having to remember all the right flags.

Moreover, if there was a way to set a default preset, then cargo could use that information to always include the flags defined by the preset in its invocations (unless a command line flag overrides it). This would:

  • Synchronize tooling that invokes cargo (bacon, rust-analyzer): They would just need to invoke cargo normally, without having to pass any additional arguments
  • Synchronize tooling that reads metadata from cargo (rust-analyzer): Cargo would automatically use the right flags read from the preset, informing rust-analyzer of the current set of features, as well as the active target, etc.

Guide-level explanation

Presets are a feature of Cargo that allow defining a set of build configurations for a project.

What is a build configuration?

Let's use an application that has both desktop and web versions as an example.

Let's also say that the desktop version also has two different windowing backends, expressed using features: window-backend-1 (default) and window-backend-2. Let's presume that despite window-backend-1 being the default, window-backend-2 works better on your system, so you would like using it as you are working on the project.

The low level way to do this is to pass --no-default-features --features=window-backend-2 to cargo every time. However, this can be difficult to remember, especially for more complex feature configurations.

The solution is to define a preset for your backend.

Create a TOML file named .cargo/presets.toml in your project's root folder:

[preset.mybackend]
default-features = false
features = ["window-backend-2"]

Now you can simply pass --preset mybackend to cargo when you want to use your preferred backend.

Let's also add the web backend:

[preset.mybackend]
default-features = false
features = ["window-backend-2"]

[preset.web]
# You can also set the target
target = "wasm32-unknown-unknown"
default-features = false
features = ["web-backend"]

Similarly, you can pass --preset web to cargo when you want to build the web version.

While this is more convenient and less error-prone than having to pass the exact flags to cargo every time, you still have to pass which preset you want to build to cargo every time. And what about tools like Rust-analyzer? Does the preset to use have to be configured separately for each tooling?

The real power of presets lie in the fact that you can set a default preset:

# Now `mybackend` is the default preset
default = "mybackend"

[preset.mybackend]
default-features = false
features = ["window-backend-2"]

[preset.web]
# You can also set the target
target = "wasm32-unknown-unknown"
default-features = false
features = ["web-backend"]

When there is a default preset set, cargo will automatically use the build configuration defined by that preset. This works for all tooling that invokes cargo, including Rust-analyzer, which queries cargo for the build configuration.

Reference-level explanation

Presets are essentially sets of build options that cargo uses in absence of a command line flag that sets that build option.

Presets are read from .cargo/presets.toml, similarly to how .cargo/config.toml is handled. Always the most specific .cargo/presets.toml is used. If a .cargo/presets.toml file is found, no further attempts are made to read and merge .cargo/presets files higher up the directory hierarchy.

Once found, the presets file is read as a TOML document, and is handled in the following way:

  1. Determine the preset to use:

    • If the file contains a default field, that preset will be used as default.

    • If a --preset <preset_name> flag is given to cargo, it will attempt to use that preset. If no preset with that name exists, cargo will stop with an error message indicating that the preset given in the argument was not found.

      If there is both a default preset, and a --preset flag, --preset overrides the default.

    • If no preset to use was determined, cargo will proceed as normally.

  2. If a preset was selected for use, then cargo will store the preset's build options as a fallback to use in absence of corresponding command line flags:

    For example in the absence of --target or --features flags, cargo will use their corresponding fallbacks.

    Any command line flag completely overrides its corresponding fallback. There is no attempt to merge a command line flag and its corresponding fallback in any way.

    However, a command line flag only overrides its corresponding fallback. For example, if a preset defines both target and features, passing a different --target flag will only override the target fallback. The features fallback will still be utilized, unless there is also a --features flag present, which would override it.

Drawbacks

  • This makes cargo slightly more complex.

    It should be a relatively simple addition though. I don't expect the implementation to add over 1000 lines.

  • Cargo has to read an additional file.

    However, this shouldn't have a significant performance impact, as it's only done once, during cargo's initialization phase.

There should be no impact to users who don't use the presets feature.

Rationale and alternatives

Other alternatives

When I initially brought up this feature suggestion, one of the responses was that this feature might have a better place in a higher level tool on top of cargo, instead of cargo itself. However, such a higher level tool doesn't exist at this time, and it's unknown if it will ever exist in such a way that integrates with tooling that currently invoke cargo (like Rust-analyzer). Without being able to integrate with tooling, such a feature would lose most of its advantages, because the user would still have to configure all their tooling manually, instead of being able to define their preferred configuration once, centrally, and have all tooling automatically use it.

Even if such a higher level tooling existed in the future, it could write to the presets file, and existing tooling wouldn't have to do anything special to invoke cargo with the right flags.

Another alternative is to implement this as a cargo wrapper (see https://github.com/crumblingstatue/cargo-presets/), which is a drop-in replacement for cargo, intercepting its command line flags, and overwriting them. While possible, it's unreasonable to expect users who want this feature to install and configure a cargo wrapper, which needs to override the original cargo executable to be usable. Cargo wrappers are also not composable, because you can only reasonably use one cargo wrapper, at least without special careful setup of an invocation chain, which a regular user can't be expected to do.

:x: Suggested alternatives that don't solve the problem

  • aliases, xtasks, native task support

    These don't integrate with tooling like Rust-analyzer, bacon, etc. Solving tooling integration is a major point of the RFC.

  • richer profiles

    Profiles live in Cargo.toml, which is a version tracked file. The user's preferred build configurations don't belong there. They are specific to the user, not the project. Moreover, there is no way to set a default profile, so tooling would know which to use.

  • Argfiles

    A file that cargo simply reads arguments from. This is unfortunately not a solution either, because not all flags apply to all comments.

Prior art

Build systems that have a configure step (for example CMake), allow generating different configurations, which can be independently built without having to respecify all the options. IDEs can even collect information about these configurations, and present a dropdown menu with selectable configurations. Qt Creator for example supports this using CMake. Rust-analyzer could potentially support a similar dropdown menu as well in editors that allow such a thing.

Unresolved questions

  • Could this be a part of .cargo/config.toml instead of .cargo/presets.toml? Would it make sense for it to be?
  • Should the feature be named presets? Would another name make more sense? build-presets is slightly more verbose, but might convey the purpose of the feature better.

Future possibilities

  • Allow more options than just default-features, features, and target
    • Setting the profile (dev/release) might be desirable.
2 Likes
2

At the point you need that, you might as well introduce a justfile/Makefile, as you might want to do other things, like changing target dir to avoid invalidating the existing one, env vars, doing some extra lints, etc. And IMO that's going to be way more maintaineable, understandable, intergate better with other tools and languages you might want to use etc. as opposed to a limited toml file.

5 Likes
3

These presets seem to essentially be profiles, but with the ability to customize more things (were you aware custom profiles are a thing already?) But I think that is exactly the direction that profiles should move in – I don't see much justification for their currently very narrow scope limited to a somewhat ad-hoc subset of code generation options. [1]

So profiles thus expanded could supersede the need for a separate "preset" feature. I guess it could become awkward if you want a Cartesian product of target ⨯ codegen profiles, such as (wasm, x64) ⨯ (dev, test, bench, release), but even if those two axes were orthogonal (ie. if there were separate "presets" and "profiles"), that might not be enough to avoid the combinatorics in some cases. Also, I'm not sure there would be any obvious, non-ad-hoc way to partition each Cargo/rustc setting into either the "preset" or "profile" bucket.

  1. It seems to me as if the things that custom profiles can currently set are pretty much exactly those that one or more of the "original" four builtin profiles wants to set. But optimally they could do much more in the future. ↩︎

2 Likes
4

Overall, I am not in favor of going down this route. I feel this is acknowledging a gap Cargo users have regarding application development but only looks at part of the problem when we should be fixing it more holistically.

Some other thoughts:

  • Currently, cargo is more plumbing and this is adding some porcelain. Tools need a way to not get the plumbing behavior for cargo commands but this isn't covered.
  • I don't think there is justification for a distinct file. When considering where this lives, we need to consider whether this is environmental configuration (determined by CWD, lives in .cargo/config.toml) or workspace/package configuration (Cargo.toml). To me, this sounds like the latter.
  • This doesn't acknowledge other alternatives, like aliases, xtasks, native task support, richer profiles, etc
    • If the concern is related to rust-analyzer support, I would want to see more on why changing rust-analyzer's config is a problem
  • For the problem with cargo's caching, we can look into alternatives for how we cache builds so we throwaway fewer builds. I suspect per-user caching might get us most of the way there and make the last leg of the work a lot easier.
2 Likes
5

How does writing a makefile solve the problem of synchronizing tooling? Rust-analyzer doesn't know about the makefile, nor does bacon, or other tools. You have to configure all those tools separately, which is a major problem that this RFC offers a solution to. Writing a makefile is not a solution.

1 Like
6

So profiles thus expanded could supersede the need for a separate "preset" feature.

Would there be a way to set a default profile, so tooling can know which profile the user currently wants to use? Would that belong in Cargo.toml? I argue it doesn't, because it's user-specific configuration, which shouldn't be tracked by Git, etc.

7

I am not sure how this proposal is going to help, but I agree that doing multi-target/feature development with r-a is a bit of a pain.

1 Like
8

The user setting what build configuration they want to use definitely does not belong in Cargo.toml, because that file is tracked by git, and you don't want the user to have to worry about accidentally committing their preferred build configuration.

None of those solve the problem of synchronizing the cargo flags used by tooling. Rust-analyzer, Bacon, etc., won't be able to use info provided by xtasks, or native task support. And richer profiles still have no way of specifying the user's preferred profile that they currently want to work with.

9

It helps because cargo itself reads the configuration file, so any tooling that invokes cargo doesn't have to do anything special. Cargo will use the right flags regardless of what invokes it.

1 Like
10

Well my problem is how much of a pain it is to switch between targets on the fly, and you'd still have to switch r-a to target a different preset somehow.

11

With default presets, you don't have to do anything special to Rust-analyzer. You just need to change your default preset, and all tooling will invoke cargo with the right flags, including Rust-analyzer, which will read the correct target from cargo. The most Rust-analyzer needs to do is reread the metadata from cargo, which could be simply done by watching for changes in .cargo/presets.toml. But can be done manually just by saving Cargo.toml, which will cause RA to reread the metadata.

12

I am open to a more holistic approach, but not one that ignores the problem of tooling integration. Any solution that doesn't involve cargo will necessitate existing tooling having to do extra work, other than just having to invoke cargo. Otherwise, they wouldn't know what build configuration to use.

It's much more tedious and error prone to have to go into Rust-analyzer's settings in your editor, and change the target string and write down the right feature configuration manually, rather than having to change a single name in a central place (The default field in .cargo/presets.toml)

And Rust analyzer isn't the only tooling that needs to be kept in sync. It's just the most prominent example. What about tools like Bacon? What about manual invocations of cargo check, cargo clippy, etc.? It's unreasonable to expect the user to make aliases for all of those commands for every build configuration, or write Makefile targets for every tooling, for every build configuration. Rust developers are used to being able to just use cargo check, cargo clippy, etc. Why break their workflow by forcing them to write Makefile targets or aliases?

13

I have a different interest here: I'd like to switch between configurations without making any versioned changes, which disqualifies RA settings as a place to do it in the case of .vscode/config.json having been added to version control. It needs to be configured in a place that's explicitly understood by the relevant tooling to be separate so it can be unversioned/ephemeral.

I wish for a system where:

  • the project announces by some means that it has multiple configurations
  • RA automatically picks up the list of configurations (this is why it needs to be some kind of protocol and not just an arbitrarily-chosen xtask)
  • I have access to a menu for quickly changing between those configurations
  • the same thing also works for users of other tools — it's not “something RA invented for itself”

On further consideration, the config.toml that the Rust project itself uses isn't too bad as a means of control, so maybe all we really need is a piece of metadata for "this project asks that you invoke its own Cargo wrapper to build it", and workflows around this can be incrementally refined from there.

2 Likes
14

This would be the case with presets.

.cargo/presets.toml would not be versioned. It would be created by the person building the project, for their own use, not to commit to version control.

And with presets, you wouldn't have to make any changes to .vscode configuration, because rust-analyzer would automatically pick up the right metadata from cargo.

15

I think the statement ".cargo/presets.toml would not be versioned", is to strong. It seems to me that there are legitimate reasons for versioning the presets. Lets say the code I'm writing needs different options for different targets as it's intended to be run on different systems. For instance an app that runs on different hardware, like bare metal embedded, or on different operating systems.

Furthermore, I'd say there needs to be a hierarchy where the "default" presets would be in /.cargo/presets.toml but a more specific version might be /cargo-presets.toml (where / represents repo root).

Also, I wonder it there should be "inheritance" like cargo.toml in workspaces

[preset.mybackend]
default-features = false
features = ["window-backend-2"]

[preset.web]
# You can also set the target
default-features.preset.mybackend = true
target = "wasm32-unknown-unknown"
features = ["web-backend"]

Or here is another possible syntax that would not require duplicating the fields directly:

[preset.mybackend]
default-features = false
features = ["window-backend-2"]

# Inherit everything from preset.mybackend and override or add other fields
[preset.web < preset.mybackend]
# You can also set the target
target = "wasm32-unknown-unknown"
features = ["web-backend"]
``
16

Just to note, inventing new [table] syntax is mostly a nonstarter; we're using TOML and diverging from TOML is a bad idea. Presets would probably use an inherits key the way profiles do.

I agree that saying profiles wouldn't be included in VCS is unlikely; I've even seen Visual Studio's "private" XML config file in VCS because people wanted settings that by default get put there (but can be put into and work from the "roaming" config), let alone less obviously user specific configuration. Instead, what makes sense is to somehow merge shared configuration from the workspace Cargo.toml and private configuration from .cargo/config.

17

Well, they wouldn't be versioned by default. It would be up to the user if they want to version it or not, but it would be unwise to version in most cases, because it contains user preferences that aren't meant to be versioned. Just as it is unwise to version .cargo/config.toml, which can contain local paths, among other things

While presets could be expanded in the future to cover default presets defined by the project, that's not in the current scope of the proposal. The user is expected to write their own .cargo/presets.toml. Projects can still put preset templates in their readme files or whatever, and the user can copy them over to their local .cargo/presets.toml.

18

I didn't understood this part of the issue, and it makes me more sympathetic to the goal.

Having said that, it feels like just piling up more things into cargo is unnecessary, and the goal can be achieved relatively simply, by just hijacking tools you want to support

We do similar things in projects I participate via Nix dev shell, which gives us full control over dev environment and doesn't require user to set up anything manually, but it's not strictly necessary to use Nix, and provide simple setup command.

What you can do is create a cargo-profiles project. I'll explain how it would work by explaining how to use it and what it does under the hood.

The user first installs it, e.g. by cargo install cargo-profiles, EZ.

The user sets it up by running:

eval $(cargo profiles env)

Under the hood cargo profiles env looks up cargo, rust-analyzer and any other supported binaries in the PATH, and then outputs

PATH="absolute-path-to-own-dir:$PATH
CARGO_PROFILES_CARGO_BIN=/path/to/original/cargo
CARGO_PROFILES_RA_BIN=/path/to/original/rust-analyzer

Inside absolute-path-to-own-dir the are symlinks named cargo, rust-analyzer, etc. pointing right at cargo-profiles binary.

This way cargo-profiles effectively and cleanly hijacked cargo, ra, and anything else it desires.

When the user calls cargo or starts and editor that starts rust-analyzer, they will call cargo-profile with a different args[0] so it can recognize what is being called and call the original binary... in a profile-desired way.

The exactly "desired way" depends on the tool, but comes down to modifying arguments passed to it, and environment used to call it. This would be based on a ./config/cargo-profiles.toml and ./config/cargo-profiles.state.toml.

After hijacking the user can call:

cargo profiles

and get a list of supported profiles with some marker which one is active, then cargo profile switch <name> to switch them, which would modify ./config/cargo-profiles.state.toml

This way you can iterate over the design and supported tools, without writing RFCs and asking anyone for permission and adding support.

You could write direnv plugin/script so the whole things loads automatically.

For IDEs you might need an extra plugin that will set it up. This is sometimes clunky, because IDEs developer ignore env handling, plugin load ordering, etc. But in principle it should work.

After you gain enough traction, figure out all the details, etc. some of the tools (e.g. RA) could possibly be convinced to respect ./config/cargo-profiles.toml and ./config/cargo-profiles.state.toml natively, and it would become a de-facto standard.

I myself would be interested in using such a thing, but don't have time for an extra project right now.

BTW. We're using a tool exactly like this, written as a simple shell script that will change CARGO_BUILD_TARGET_DIR for cargo commands building a subset of packages (build -p <pkg>, etc.), to avoid invalidating ./target due to different feature/dependency set. I was consider asking community for something like this behavior built-into cargo, but it works perfectly as a shell script for our own use.

1 Like
19

Yeah, I seem to have a problem with making myself understood, so I don't think writing RFCs is for me honestly.

There is no need to override anything other than cargo, which is something I'm already doing (sans the env part). All tooling that invokes cargo works automatically without changes, once cargo knows about presets.

You're right, since this pre-RFC has mostly only received arguments against it (most of which don't understand the problem I'm trying to solve), I'm closing this pre-RFC effective immediately.

Maybe someone else can write a better RFC in the future, but for now, a cargo wrapper will suffice for me.

20

FWIW. I think I just didn't had the time to read through it carefully. I don't think there's anything wrong with RFC per se.

Haha. So you already doing exactly the thing. I was not aware of it. I will definitely consider using and contributing!

I think the tool like this is good enough and marketing it a bit in the ecosystem might be a best approach to go about it for now.

1 Like