# \[Pre-RFC\] Cargo - custom named profiles

**URL:** <https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538>\
**Category:** cargo\
**Created:** [March 1, 2019, 5:50pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538 "2019-03-01T17:50:03Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![da-x](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/da-x/32/3965_2.png) [@da-x](https://internals.rust-lang.org/u/da-x)\
**Post date:** [March 1, 2019, 5:50pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/1 "2019-03-01T17:50:03Z")

</div>

I would like to proposed _named_ custom profiles in Cargo.

( **EDIT** : RFC opened: [RFC 2678](https://github.com/rust-lang/rfcs/pull/2678))

# Overview

Past proposal to increase flexibility of Cargo’s build flags for crates within a single `cargo build` invocation, has resulted in [RFC 2282](https://github.com/rust-lang/rfcs/blob/master/text/2282-profile-dependencies.md), which adds the flexibility of changing attributes of specific crates under one of the default profiles. However, useful as it is, it does not allow for a full custom profile name definition that can have its own separate directory under `target`, in parallel to `debug` and `release`.

# Motivation

First, it I think it would be worth to note that I don’t regard this extension as making the previous `overrides` extension unnecessary. Rather, I think it is in fact orthogonal and complementary to it.

The motivation for having named custom profiles is to be able to easily throw everything under a custom compilation mode, which is chosen when Cargo is invoked from the command-line, just like `--release` is chosen. For example, if suppose I am frequently comparing between both a release build and a super-optimized release+LTO build, I would like Cargo to having two separate `target/` directories, e.g. `target/release`, and `target/release-lto`, for which the binaries and incremental compilation is managed separately. This is so that I can easily switch between the two modes without penalty.

Here’s an example actual real-world user: [tikv/issue/4189](https://github.com/tikv/tikv/issues/4189)

# Guide-level explanation

This proposal’s design is simple - let the user define profiles under new names, keeping the original names reserved, and allow the new profiles to inherit from existing ones. The exact `Cargo.toml` structure can be as such:

```
[profile.custom.release-lto]
inherits = "release"
lto = true

```

Passing `--profile` with the profile’s name to various cargo commands will resolve as if we used the inherited profile, with the overrides specified in the inheriting profiles, and all outputs will go to a different directory.

```
$ cargo build
$ cargo build --release
$ cargo build --profile release-lto
$ ls -l target
debug release release-lto

```

Having this result in a separate directory is important, because the intended use of this feature is usually to affect the build flags for all dependent crates anyway, and because we cannot currently have the same crate be built with different flags under the same profile.

**Unanswered questions**

- Is there a cleaner way to accomplish this now?
- Bike-shedding whether we call this ‘inherit’ or something else (`include`?).
- What restrictions to put on profile names
- Should `cargo clean` behave regarding the custom profile output directories

---

<div class="post-metadata">

**Author:** ![ehuss](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@ehuss](https://internals.rust-lang.org/u/ehuss)\
**Post date:** [March 2, 2019, 5:40am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/2 "2019-03-02T05:40:47Z")

</div>

Thanks for posting this!

For context, the original discussion had something similar, but was deferred for a broader workflow integration. However, the Cargo team discussed this and it seems like a reasonable thing to do in the short term. Workflows are unlikely to make significant progress soon, and I personally think it would be reasonable if a workflow definition just specified a default profile.

I’d like to hear from @matklad and @manishearth about their thoughts. Overall I think this would be useful.

I have a bunch of questions/concerns, but not all of them need to be answered now.

- What is the purpose of using the word `custom` in the profile key name? Do custom profiles have a separate namespace from built-in profiles? If so, does that mean you can’t say `--profile=release`? If they share a namespace, that would make adding new built-in profiles a problem. It can also make adding new things to the `target` dir difficult.

- Profile names would collide with rustc target names. In practice that’s unlikely ever to be an issue, but something to be aware of.

- It might be good to consider the future of the test/bench profiles. I don’t immediately see an impact, but I think custom profiles will contribute to how they are confusing and special. There is some desire to deprecate them, but I don’t completely agree with that. I think it would be interesting to keep them around, but change their defaults to `inherits = "dev"` (or `release` for bench), and reduce how special they are.

- Can an override inherit another profile? It seems reasonable, but just want to be clear.

- `check` and `rustc` have a legacy `--profile` flag. How should custom profiles interact with that? My preference would be to rename the existing flag to `--mode` and then perhaps have some sort of backwards-compatibility. That may be tricky.

- I am working on adding a `build` profile ([https://github.com/rust-lang/cargo/pull/6577](https://github.com/rust-lang/cargo/pull/6577)). This may have problematic interactions with custom profiles. The issue is when a project has a large number of shared dependencies (between regular and build dependencies) that it can significantly increase initial build time. The intent is that if a user wants to share artifacts, they can specify `[profile.dev.build_override]` and `[profile.release.build_override]`. If there are even more profiles, it would have to be specified for each one, which sounds awkward. I think that interaction needs to be figured out.

- What should the `PROFILE` environment variable say for custom build scripts?

- What should the “build type” be for the `[finished]` line say? Currently it is “release” or “dev”. The current design has caused some confusion, because a single command may use multiple profiles.

- Should `cargo test --profile bar` build the entire artifact and all deps with the `bar` profile, or should it continue to use the mixed test/bar profiles? I would prefer if it only used `bar`, and start to move away from the weird hybrid approach currently used.

---

<div class="post-metadata">

**Author:** ![matklad](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/matklad/32/12266_2.png) [@matklad](https://internals.rust-lang.org/u/matklad)\
**Post date:** [March 2, 2019, 9:28am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/3 "2019-03-02T09:28:30Z")

</div>

I like this! I do however have couple of comments on implementation side of things (which I am less familiar with than @ehuss)

- I feel we need to figure out what to do with bench & test profiles. I’d game for removing them, `cargo test` and `cargo test --release` should behave like `cargo build` and `cargo build --release`. Making them just usual profiles which inherit from dev/release seems fine as well, but I’d go for complete removal, if it is feasible.
- `target` directory is complicated. Given the ideas about “global binary cache”, I think it make sense to maybe radically re-think how target is structures? The problem of #6577 where `build` profile does not work well because `debug` and `release` are different folders is a good case for this. 30-seconds thinking proposal would be to introduce `target/.cache` as a opaque directory where Cargo keeps all intermediate artifacts, and keep `target/{profile}` or `target/{tripple}/{profile}` solely for the end artifacts (what unstable `--output-dir` does today). This hopefully should allow sharing dependencies across profiles? (oh, that’s basically [https://github.com/rust-lang/cargo/pull/6577#issuecomment-459415283](https://github.com/rust-lang/cargo/pull/6577#issuecomment-459415283) :))

---

<div class="post-metadata">

**Author:** ![kornel](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kornel/32/2711_2.png) [@kornel](https://internals.rust-lang.org/u/kornel)\
**Post date:** [March 2, 2019, 5:34pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/4 "2019-03-02T17:34:44Z")

</div>

I think existing behavior can be ported to the concept of custom profiles:

- `test` profile inherits the `debug` profile,
- `cargo test --release` is a `test` profile that inherits the `release` profile,
- `bench` inherits the `release` profile.

I’ve started refactoring this code in Cargo a while ago. `test` being a boolean in lots of places was definitely sub-optimal. But I never finished it, because of the vague notion that the team thought about doing something better instead.

---

<div class="post-metadata">

**Author:** ![da-x](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/da-x/32/3965_2.png) [@da-x](https://internals.rust-lang.org/u/da-x)\
**Post date:** [March 2, 2019, 8:57pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/5 "2019-03-02T20:57:57Z")

</div>

I’ve gathered some answers to questions raised:

- **release/debug folders** : I am under the assumption that they are going to stay (I support @alexcrichton’s comment about this). Also, regardless of whether we make builds faster by global or local caching via improved file hashing and other methods (i.e. ‘shared artifacts’ between ‘dev’ and ‘release’), I think that the the user-facing aspect of custom profiles would be more focused on the final target outputs.
- **Namespace for profile names** : It’s not by chance - I like the idea of a separate namespace for custom profiles. Perhaps these should also be specified using `--profile custom.<name>`, rather than directly.
- **Target directory** : Perhaps we can put the output of all custom profiles under `target/custom/<profile>`, to avoid directory naming collisions.
- **test/bench profiles** : Without minding how they are being regarded _internally_ in Cargo, as a user I have always regarded these two as execution modes rather than build profiles. To illustrate, while having custom profiles, I would like to test/bench my `release-lto` and `release` separately and independently.
- **Overrides** : I think it may create confusion if we add ‘inherit’ directive to overrides, because otherwise the Overrides and Custom Profile features can remain orthogonal.
- **Obsolete rustc --profile** : I agree this needs a migration path to `--mode`.
- **Build profile** : Might not be an issue - for example if a profile inherits from `release`, then it would also inherit all of its `[profile.release.build_override]` declarations, so no repeating would be necessary.
- **`PROFILE` env** : ( **EDITED** ) should contain the output directory name rather than the profile name.
- **Build type reported** : I guess that the name of the specified profiled should be printed.
- **Separation of artifacts**. Before the shared artifacts change is implemented, I suggest that `cargo <command> --profile <name>` would not modify any files that are _not_ under `target/custom/<name>` (but it should be able to _read_ from a global artifact cache once it is implemented, in the future). Perhaps this way we improve predictability of how Cargo works - the user can expect how long he or she’s dev cycle is going to last based on what things are cached in which profile sub directory.

---

<div class="post-metadata">

**Author:** ![ehuss](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@ehuss](https://internals.rust-lang.org/u/ehuss)\
**Post date:** [March 2, 2019, 9:01pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/6 "2019-03-02T21:01:47Z")

</div>

> [@matklad](#):
>
> Making them just usual profiles which inherit from dev/release seems fine as well, but I’d go for complete removal, if it is feasible.

The reason I'm reluctant to remove them is that there are some situations where there isn't a clear default. For example, in the original motivation there is a "release" and "release-lto" profile. Which one is appropriate for `cargo bench`? Or maybe normal "dev" profile is intolerable without aggressive optimization, but the user wants something different for `cargo test`. Sometimes projects don't have a clear answer. But I don't think it is critical to make a decision now.

> [@matklad](#):
>
> opaque directory where Cargo keeps all intermediate artifacts

Yea, that's essentially what I've implemented in [Unify build directories. by ehuss · Pull Request #6668 · rust-lang/cargo · GitHub](https://github.com/rust-lang/cargo/pull/6668). I'm concerned that this will break a lot of projects and workflows. I want to make sure that the affected use cases have some way to work correctly. Cargo may need to provide more information (like asking it for directories and such). No matter what, I think the transition will be bumpy for some people.

---

<div class="post-metadata">

**Author:** ![matklad](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/matklad/32/12266_2.png) [@matklad](https://internals.rust-lang.org/u/matklad)\
**Post date:** [March 2, 2019, 9:14pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/7 "2019-03-02T21:14:36Z")

</div>

> [@ehuss](#):
>
> The reason I’m reluctant to remove them is that there are some situations where there isn’t a clear default.

Yeah... I guess it might make sense to simultaneously remove test/bench profiles and introduce custom profiles? Then, if you really make use of a custom test profile, you can now run ‘cargo test —profile test’, where test is just a usual custom profile.

---

<div class="post-metadata">

**Author:** ![Manishearth](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/manishearth/32/50_2.png) [@Manishearth](https://internals.rust-lang.org/u/Manishearth)\
**Post date:** [March 8, 2019, 7:25pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/8 "2019-03-08T19:25:01Z")

</div>

> [@ehuss](#):
>
> I’d like to hear from @matklad and @manishearth about their thoughts. Overall I think this would be useful.

> [@ehuss](#):
>
> Can an override inherit another profile? It seems reasonable, but just want to be clear.

I'll mention that the original design for [2282-profile-dependencies - The Rust RFC Book](http://rust-lang.github.io/rfcs/2282-profile-dependencies.html) included custom profiles designed similarly to these, so that overrides would be a matter of "using" profiles as overrides instead of having their own profile-like key structure. However at the time due to the workflow concerns this stuff was postponed.

I don't really have any comments on this RFC, the design seems fine. I mostly agree with @ehuss and would like to mention that I also think that we should carefully think about test/bench. I have a slight preference towards not putting these under `profile.custom` and `custom/`, but it's nbd.

---

<div class="post-metadata">

**Author:** ![ehuss](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@ehuss](https://internals.rust-lang.org/u/ehuss)\
**Post date:** [April 2, 2019, 12:17am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/9 "2019-04-02T00:17:18Z")

</div>

@da-x Can you collect this into an RFC and post it on the RFCs repo? (Some of the detailed points can go into “Unresolved questions” if they don’t have an obvious answer.)

I think we fundamentally want to move forward, there’s just some minor details to work out. The only real concern I have is namespacing. My preference would be to have a flat namespace and not worry about a special “custom” identifier. Restrictions can be placed on the custom name so that if absolutely necessary, we can work around it. I’m not concerned about Cargo adding new profiles in the future, and I’m also not too concerned about conflicts with target names.

---

<div class="post-metadata">

**Author:** ![da-x](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/da-x/32/3965_2.png) [@da-x](https://internals.rust-lang.org/u/da-x)\
**Post date:** [April 2, 2019, 7:12am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/10 "2019-04-02T07:12:39Z")

</div>

@ehuss - sure! I’m intending to get it done by the end of this week.

---

<div class="post-metadata">

**Author:** ![cliff](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cliff/32/2650_2.png) [@cliff](https://internals.rust-lang.org/u/cliff)\
**Post date:** [April 2, 2019, 1:53pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/11 "2019-04-02T13:53:38Z")

</div>

An alternative to namespacing would be to require a particular naming convention for custom profiles, like all custom profiles have a colon. This allows a sort of ad-hoc namespacing like `custom:release-lto`, `client:dev`, or (unnamespaced) `:release`. Any punctuation character would do. Period or hyphen are options. Hyphen would feel less namespacey, if that were preferred, though leading hyphen leads to potential confusion with command line options, and leading period creates hidden directories, so in those cases, perhaps a rule that the first character must be a letter would be useful.

---

<div class="post-metadata">

**Author:** ![da-x](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/da-x/32/3965_2.png) [@da-x](https://internals.rust-lang.org/u/da-x)\
**Post date:** [April 4, 2019, 3:08pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/12 "2019-04-04T15:08:18Z")

</div>

RFC opened: [RFC 2678](https://github.com/rust-lang/rfcs/pull/2678).

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [July 3, 2019, 3:08pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538/13 "2019-07-03T15:08:23Z")

</div>

Closed on July 3, 2019, 3:08pm UTC

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
