Pre-RFC: Allow packages to specify a set of supported targets

[epage]

@carloskiki

This field consists of an array of strings, where each string is an explicit target-triple, or a cfg specification (as for the [target.'cfg(**)'] table). The supported cfg syntax is the same as the one for platform-specific dependencies (i.e., cfg(test), cfg(debug_assertions), and cfg(proc_macro) are not supported).

A rationale that is not given is why is this an array of strings? The reason I originally suggested that was for non-cfg support. If cfg is supported, why do we have it?

[kornel]

What's the proposed syntax for cfg support? If that's just supports = "cfg(x)", then I see a couple of drawbacks:

The top-level all vs any may be unintuitive. You want to say you support all of these platforms, but the cfg needs any().

Complex cfg() gets difficult to read beyond a couple of platforms, especially if they need extra nesting to refine the target (e.g. exclude soft float or require crc extension). It would need multi-line string and neat formatting. TOML formatters can easily format an array of strings, but pretty-printing a big cfg() string would need special support.

The current target.'cfg' is a one cfg per platform, rather than one cfg for all platforms. An array of strings with supported cfgs could closely resemble cfgs in target.cfg.

[epage]

To be clear, I'm not for or against an array at this point; we just need a rationale to outweigh the implicit any

[epage]

@carloskiki

To improve usability, two extra relations are defined:

• cfg(target_os = "windows") ⊆ cfg(target_family = "windows").
• cfg(target_os = <unix-os>) ⊆ cfg(target_family = "unix"), where <unix-os> is any of ["freebsd", "linux", "netbsd", "redox", "illumos", "fuchsia", "emscripten", "android", "ios", "macos", "solaris"]. This list needs to be updated if a new unix OS is supported by rustc's official target list).

If you want to get this through more smoothly, it might help to move this to a future possibility. Including this will require coordination with T-compiler and opens the door for what subset rules are allowed vs which ones aren't.

[DragonDev1906]

Another concern about using cfg: Doing so makes it really hard to enumerate/list all "supported" variants. Places where this would be useful:

• Filtering crates based on whether they support a specific (or set of) targets. While this would be possible with cfg, doing so would be harder and its thus less likely to be done
• Doing a task for every supported configuration. For example running all tests on all targets or compiling a binary for all "supported" targets. With a simple list that doesn't contain cfg this is really easy, but as soon as cfg is involved you additionally need to bother with parsing it and it can make it hard to see which configurations are included. You may have to iterate a list of all possible target-tripple + feature flags + ... combinations just to enumerate them or you need more complex logic to reduce that set.

I think the usefulness of easy enumeration and ease of parsing/doing so should not be underestimated. cfg is good at saying yes or no, but I don't think that's enough here.

Personally, I think doing it in toml is better than using cfg, for example:

[[lib.supported-configurations]]
targets = ["wasm32-unknown-unknown", "wasm32-wasi"]
required-features = [] # Enabled automatically when compiling for that target

[[lib.supported-configurations]]
# Can be extended with wildcards
target = ["*-linux-*"]

[[lib.supported-configurations]]
# If there is a desire to further group targets together
# (to reduce the amount of typing)
target_os = ["windows"]

Assuming per-target required features should be represent-able configurations probably makes more sense and avoids the target renaming discussion

Another advantage of doing it this way would be that it can be extended more easily. For example: With cfg you might have a hard time adding an extra (optional) support_level field in the future (similar to the levels of rust target support). Even with the basic list-of-strings such extendability is not possible or at least harder to do in a backwards compatible way.

[kornel]

If you use *-linux-* as a supported target, using it will require another parser. As soon as you will need to express something more, like "Linux except that one very weird CPU", you'll need the cfg() again, or reinvent cfg with a custom glob microsyntax.

I think full power of cfg() is necessary to let crates describe their requirements as precisely as possible, to maximize the chance of them correctly allowing/disallowing future platforms. Otherwise it may end up being a problem similar to the User-Agent, where old crates only allowed platforms that existed at the time, and new platforms don't work just because they're not on the list. We could end up with aarch64-newplatform-linux-compatible-like-musl

I think difficulty of working with arbitrary cfg() could be mitigated with tooling, e.g. there could be rustup target list --filter cfg(…) or cargo metadata could include expanded lists.

[carloskiki]

Wow, thank you for the lengthy feedback!

It may take a bit before I answer this comment, I want to make sure I understand and cover everything in it.

[carloskiki]

Agreed. I will be moving this to a future possibility.

[carloskiki]

@epage

@carloskiki

• "cargo-target": The part of a package that is built ([lib], [[bin]], [[example]], [[test]], or [[bench]]).
• "target": The architecture/platform that rustc is compiling for.

Would be good to link to the glossary and either use the terms from there (cargo-target, target triple) or my own spin which is build-target and platform-target.

I will go with your spin: build-target and platform-target. The problem I have with "target-triple" is that I find myself sometimes wanting to express a subset of target-triples, not only a specific target-triple. I will also link to the glossary.

@epage

@carloskiki

Compatibility with dependencies

This never says what happens. Is it an error? A warning? A lint?

@carloskiki

This is enforced only upon publishing (and thus only for crates). Since supported-targets is only useful for dependents of a library or binary, enforcing this at build time is not necessary and overly intrusive.

Since this gets into terminology, there are "crates" which are build units and there are .crate files which are published.

This can matter for more than just published packages, like git dependencies or even path dependencies in a monorepo.

Also, publish-time is too late to notify people of a problem as that is the very tail end of a release process. Feedback needs to happen during development. Also, irreversible actions may have already been taken as part of the release process.

As you mention later, this should be an error, just like everything should actually be an error and not a lint.

I also did not properly analyze the issue, as you demonstrate. My idea was that having this error at build time would be annoying especially for one-off projects that are not intended to be shared or published. Also, once a crate adds restrictions on its supported targets, every dependent must also use the feature. I appreciate however that this must be done, so I believe this should be run at build time as well. Do you have any thoughts on this?

@epage

@carloskiki

Ignoring builds for unsupported targets

When the target used is not supported by the cargo-target being built, the cargo-target will either be skipped or a lint will be raised, depending on how cargo was invoked. cargo behaves identically to when the required-features of the cargo-target are not satisfied.

Note: required-features is not applicable to libraries, while supported-targets is. Nevertheless, libraries behave the same way as other cargo-targets.

This seems like the place to specify this behavior or at least provide guidance on when each is chosen. Leaving it this vague prevents meaningful discussion to work out the design.

I was not able to find a good place where this behavior was specified, the cargo book does not explain the decision process. I was able to find this in the source code:

/// .... Generally, targets specified by name (`--bin foo`) are
/// required, all others can be silently skipped if features are missing.

For libraries, this means that running with --lib would fail if the target is not supported, and otherwise it would silently skip the build.

Is this enough, or should I dig deeper?

@epage

@carloskiki

Detecting unused dependencies

Isn't this the same check as "Compatibility with dependencies" and one says it should lint as incompatible while this says it is unused?

So "compatibility with dependencies" is about making sure that a crate cannot support a target which one of its dependencies does not support.

"Detecting unused dependencies" is about purging dependencies that are never used from Cargo.lock. This is both for the crate itself having unused dependencies because the crate has [target.'cfg(..)'.dependencies] tables that are never matched, and for transitive dependencies behind these tables (the crate has no control over this).

@epage

I don't think that is enough of a difference to justify a different name.

I do not have a preference on the name. Since others seemed to like supported-targets that was the term used, but I can see the mnemonic value of required-targets following the convention of required-features.

One thing I may do is start a poll once I open the RFC, so that we can get a better idea of what people prefer.

@epage

@carloskiki

Not using cfg

Using cfg complicates the implementation, and may require a substantial amount of calls to rustc to check target-cfg compatibility. Some alternatives are discussed here along with their drawbacks.

This leaves off a big problem with cfg: if we support this in package and want to rely on this for trimming the Cargo.lock file, things may get messy.

This will be added.

@epage

@carloskiki

How do we make users bypass the lint (probably with some --ignore-** flag)?

imo this is my problem with calling it a lint: Cargo only offers static lint control and I don't see it making sense to set this and override it with static lint control.

It can just be an error instead, even if there is a flag for overriding it. The question is, do we need to override it?

As I said earlier, I think having this as an error is better, we can always transform it into a lint later. I can see however that people may want to override this. Some crates will be too strict. I understand that technically one can always fork the crate, but being forced to fork a crate to change a single line in Cargo.toml is not ideal.

@epage

Something this leaves out from the Alternatives is why not also support this in package which would apply to everything and could be easier to use in most cases and be to apply to dependency resolution.

The rationale to having this per cargo-target was that it is more expressive. I can imagine some people might have examples, tests, or benchmarks that use different targets. I also have had in the past binaries (requiring desktop OS) that were used as tools for a bare-metal project in the same package. I appreciate that having the feature at the [package] granularity is not a blocker, as people can always separate their projects into different packages inside a workspace. This also makes the interaction with dependency resolution easier, as dependencies are solved at the package level.

It was also discussed previously that having this feature both at the [package], and at the cargo-target level, where cargo-targets can override the [package] level, would be beneficial. Since I'm not aware of any other fields in cargo doing this, I did not include it. Would you rather have this at the [package] level, or have it at both levels?

[carloskiki]

If there was a cfg option like cfg(target = "x86_64-linux-..."), then I could see how using only cfgs could work. I decided on a list of strings because it was simple, and you already proposed it previously. I have not taken the time to think through this thoroughly, I am always open to other propositions.

I think the proposition by @DragonDev1906 looks nice (minus the glob operator):

@DragonDev1906

[[lib.supported-configurations]]
targets = ["wasm32-unknown-unknown", "wasm32-wasi"]
required-features = [] # Enabled automatically when compiling for that target

[[lib.supported-configurations]]
# Can be extended with wildcards
target = ["*-linux-*"]

[[lib.supported-configurations]]
# If there is a desire to further group targets together
# (to reduce the amount of typing)
target_os = ["windows"]

But it also feels way too complicated for what the feature is.

I will try to find a good rationale and add it to the RFC.

[epage]

@carloskiki

I was not able to find a good place where this behavior was specified, the cargo book does not explain the decision process. I was able to find this in the source code:

/// .... Generally, targets specified by name (`--bin foo`) are
/// required, all others can be silently skipped if features are missing.

For libraries, this means that running with --lib would fail if the target is not supported, and otherwise it would silently skip the build.

Is this enough, or should I dig deeper?

Saying "this will follow the same logic when required-features is not satisfied" will work. Probably also good to call out the cases this affects

• dependencies on [lib]
• (unstable) dependencies on artifacts
• --all-targets
• all of a kind flags (e.g. --bins)
• specific build-targets (e.g. --bin foo)

@carloskiki

So "compatibility with dependencies" is about making sure that a crate cannot support a target which one of its dependencies does not support.

"Detecting unused dependencies" is about purging dependencies that are never used from Cargo.lock. This is both for the crate itself having unused dependencies because the crate has [target.'cfg(..)'.dependencies] tables that are never matched, and for transitive dependencies behind these tables (the crate has no control over this).

I think I'm still confused.

Is "compatibility with dependencies" about (package.required-targets | target.<cfg>) intersects dep.required-targets

While "unused dependencies" is about package.required-targets intersects target.<cfg>?

@carloskiki

for transitive dependencies behind these tables (the crate has no control over this).

Cargo uses a concept call cap-lints which means that warnings from transitive dependencies do not affect your build.

@carloskiki

I do not have a preference on the name. Since others seemed to like supported-targets that was the term used, but I can see the mnemonic value of required-targets following the convention of required-features.

One thing I may do is start a poll once I open the RFC, so that we can get a better idea of what people prefer.

I have little confidence in using a poll for making a decision like this.

@carloskiki

The rationale to having this per cargo-target was that it is more expressive. I can imagine some people might have examples, tests, or benchmarks that use different targets. I also have had in the past binaries (requiring desktop OS) that were used as tools for a bare-metal project in the same package. I appreciate that having the feature at the [package] granularity is not a blocker, as people can always separate their projects into different packages inside a workspace. This also makes the interaction with dependency resolution easier, as dependencies are solved at the package level.

My main care about is that its discussed in the proposal.

For [package]:

• Easier to to apply to all targets
• Less error prone because auto-discovered build-targets will be "all targets" when that might not be intended.
• Can look in one place

There is also the question of whether multiple build-targets or multiple packages is the right model in general and for the use cases here that need the "expressiveness".

At a first glance, I lean towards "both places"; I wanted to make clear its not an automatic win one direction or the other.

@carloskiki

It was also discussed previously that having this feature both at the [package], and at the cargo-target level, where cargo-targets can override the [package] level, would be beneficial. Since I'm not aware of any other fields in cargo doing this, I did not include it. Would you rather have this at the [package] level, or have it at both levels?

I do not see build-targets overriding the package but being more specific than it, much like required-features is more specific than what is in [dependencies].

@carloskiki

If there was a cfg option like cfg(target = "x86_64-linux-..."), then I could see how using only cfgs could work. I decided on a list of strings because it was simple, and you already proposed it previously. I have not taken the time to think through this thoroughly, I am always open to other propositions.

Again, the important thing is to cover it in the document so there is a point of focus for discussion, rather than what my individual preferences are.

[Eh2406]

Hi, maintainer of Cargoes resolver and the PubGrub crate. As changing the resolver is left as a Future possibility, I have been lurking in this thread without sidetracking the conversation with the resolver details.

@carloskiki

In the future, this could be implemented in pubgrub using constraints.

How involved the resolver changes are going to be depend on what semantics we want to use. I don't see a semantics where just adding "constraints" will be all the changes that are needed. (But it is entirely possible I missed something.) Can you elaborate what you had in mind with this line?

[carloskiki]

I have very little experience with dependency resolution or pubgrub, and this was more of a "just put it out there statement," until I properly flesh out the Future Possibilities section. What I said is very likely wrong. I was trying to see if the feature could be implemented into pubgrub using the Version trait, and it felt like it was not enough. I was looking for other options, and the "constraints" described in the linked issue felt relevant.

As you said since this is left as a future possibility, I don't want to bother and detail out what it would take to implement it in the resolver. However when I rework this section, I will make sure to not state anything without being sure of it.

[carloskiki]

I have updated the Pre-RFC to touch on what was discussed. Previous versions are available on github.

Most notably, the feature is now usable at [package] level, and is no longer usable at the cargo-target level (this is left as a future possibility).

This document feels mature enough for an RFC, although I have never written one before so please correct me if I am wrong. The only thing left would be the Prior Art section, but I do not have a ton of in depth experience with build tools, so I am not aware of any other solutions to the problem this RFC fixes. If anyone knows of other solutions, please let me know!

[gibbz00]

Here are some potential additions for the Prior Art section / interactions with related efforts that may be worth mentioning.

--target and build.target

How should cargo behave if one or multiple --targets or .cargo/config.toml build.targets is not present in supported-targets?

per-package-target

My primary interest for this RFC is in how it may push per package targets closer to stabilization. (default-target makes it possible to run the usual clippy, fmt, test etc. as workspace commands in projects that contain couple of target specific crates. Development in such projects not using this feature is painful as one needs to call these commands "manually" for each package, especially when rust analyzer does not support such a setup either. This is quite common in embedded domains, ex. embassy.)

Should package.default-target and package.force-target be rewritten to only allow values present in the resolved set of supported-targets?

Now to the part that is slightly off-topic. Let me know if this should be moved elsewhere. This is my first time posting here.

package.force-target should perhaps be removed in its entirety as alluded to here. I would also stand behind this unless I've misunderstood its original purpose.

I take it from the tracking issue that the goal of force-target was to enable building for multiple targets at the same time? Some through --target and one with force-target. Some people have already expressed a want for this to instead be a list of target triples. My (naive) suggestion would be to replace this with a --supported-targets flag. Obviously, there needs to be a warning when the set of supported targets is practically unbounded. Those that wish to build only a strict subset of supported-targets in one command could initially rely on -Zmultitarget. I don't think default-target should simply be made into a list that corresponds to the top-level all support for this. At least not for now. Granted, these points may better belong in a follow-up RFC.

[carloskiki]

I have posted the RFC!

Thank you all for the the feedback.

[binarycat]

as an former nix developer, this makes me wary. i find developers will far too often only list a few common targets as supported, even though other targets may have work fine.

or, they become careless, and say they support targets they actually don't.

or! in the case of stage 3 targets, they support a target, but then there's a regression, so that target no longer works.

[mathstuf]

I agree…unless there is a specific reason a crate should be platform specific (e.g., windows-api, some Linux-specific syscall wrapper, etc.), gate-checking platforms leads to the problems that have been seen in places like C and C++ compiler support detection macros where new compilers and versions end up falling into the #error unknown compiler hole. New frontends end up "emulating" other compilers in order to not have to patch "everything". This is why (almost) all compilers define __GNUC__ and GCC is basically never allowed to specify an "I am GCC" preprocessor macro: as soon as they do, other toolchains will provide it to get past these "ID check" bouncers. Instead, compilers need to be detected by "well, it doesn't say what kind of clang it is (IBM, Apple, Fujitsu, IntelOneAPI), so it must be upstream" kinds of logic.

[Noratrieb]

The RFC has been posted, all feedback should be directed at the RFC instead.