Pre-RFC: Cargo Target Features

[sorear]

For things like AVX and AES-NI in x86-only crates, I oppose this proposal and would strongly prefer library crates to do runtime feature detection using CPUID. No extra work for Cargo or the end application, and gives you wide binary compatibility as a perk. AFAIK, no recent user-mode x86 extension has provided fundamentally new capabilities, so you can always provide a fallback using older instructions and/or pure Rust code.

This doesn’t work for SSE2, because SSE2 affects the ABI, but at this point I think it’s reasonable to assume that by default (as Rust currently does).

This also doesn’t work as well for the supervisor-mode extensions that do provide new capabilities, but you can still have a runtime check and return an error.

If you plan on exporting very short functions that do this, it would be a good idea to cache the relevant feature flag in a static mut AtomicUsize, since CPUID is a serializing instruction. Also this runs into trouble if you have a multi-socket system with CPUs of different families/generations, because the CPUID could run on the newer CPU and the process rescheduled to the older one, but runtime feature testing is common enough in non-Rust programs that that ship sailed long ago.

So, what can we do to Rust to facilitate and encourage runtime feature testing?

Finally, remember that not all the world is x86, and we should design Rust to facilitate using the pure-Rust option on non-x86 CPUs (and I don’t just mean ARM – I’m also talking about the architecture neither of us has heard of that will be widely deployed 15 years from now.)

[ruudva]

For things like AVX and AES-NI in x86-only crates, I oppose this proposal and would strongly prefer library crates to do runtime feature detection using CPUID. No extra work for Cargo or the end application, and gives you wide binary compatibility as a perk.

You would still need to pass the -C target-feature=+feature, otherwise LLVM will not be able to compile the code, even if at runtime you decide not to use the feature. If you want to do runtime feature detection that is great, but this is up to the author to decide. If you know beforehand that your program is only going to run on a specific system, then it does not make sense to implement a fallback.

You do bring up an interesting point though, because runtime feature detection means that a target feature can be required for compilation (i.e. you must pass -C target-feature=+feature to be able to compile at all), without making the resulting binary incompatible with systems that do not sport the feature.

This also doesn’t work as well for the supervisor-mode extensions that do provide new capabilities, but you can still have a runtime check and return an error.

Rust tries very hard to turn runtime errors into compile errors. On the other hand, with the runtime check you can at least print a nice error message. If the instructions are used unconditionally, the program will simply crash. If you do know the system you are going to run on, then you might not want to pay the cost of a runtime check.

I think that we cannot rule out either static features or dynamic detection, the proposal should be able to facilitate both.

Finally, remember that not all the world is x86, and we should design Rust to facilitate using the pure-Rust option on non-x86 CPUs (and I don’t just mean ARM – I’m also talking about the architecture neither of us has heard of that will be widely deployed 15 years from now.)

I happened to pick x86_64 examples here, but the proposal is not specific to any architecture. Do you see any issues here?

[ruudva]

So, what can we do to Rust to facilitate and encourage runtime feature testing?

I think that this can all be done by libraries. For instance, a crypto crate could query the CPU features and dispatch on the features in an encrypt function, and users of the crate automatically get the most efficient implementation without worrying about anything.

This only works well for relatively large functions though. For things like adding f32x8s from the simd crate, you want these to be inlined; feature detection for every add is far to expensive. Then detection should happen at a higher level, but now it gets complex quickly: ideally the simd crate would provide a fallback implementation of Add and the user of the crate should not have to duplicate any code. But it is the user’s code that will have to be compiled twice.

[alexcrichton]

Thanks for posting this! I’ve certainly had thoughts on this before, and I’ll try to transcribe them for now as well

Cargo works pretty hard to make it possible that the top-level package has complete control over the build in terms of flags and codegen. For example the [profile] sections of dependencies are ignored so the top-level decides things like codegen-units, optimization level, debuginfo, etc. I’m not sure that we want to encourage a required list of target features because this is the opposite of this pattern. The top-level package no longer has that level of control and you may not realize that you shouldn’t have been generating AVX instructions until too late.

Along those lines I might be hesitant to move on this just yet and prefer to keep it under wraps as long as possible. Support for RUSTFLAGS landed recently which should allow passing -C target-feature=... to all compilations, but beyond that I agree with @sorear that the best practice here anyway is to write code which compiles and works without these target features enabled, and it seems like Cargo should encourage that.

[jethrogb]

no recent user-mode x86 extension has provided fundamentally new capabilities

This is not true. TSX and SGX come to mind. It is possible to write a library that defines new synchronization primitives that can only be used if TSX is enabled. Thus, the dependency on TSX should trickle up the dependency tree to the final build product. Likewise, using a library that depends on SGX without SGX is either not possible or removes the security guarantees that that library was aiming to provide.

[ruudva]

I’m not sure that we want to encourage a required list of target features because this is the opposite of this pattern. The top-level package no longer has that level of control and you may not realize that you shouldn’t have been generating AVX instructions until too late.

This is why I propose that the top-level project specifies target-features, and the top-level project and dependencies get compiled with these features. But authors might want to write code that depends on a certain target feature, and if they do not provide fallback code, that target feature is definitely required — there is no getting around that. Hence, when one of the dependencies requires a feature that was not selected by the top-level project, Cargo should refuse to build.

the best practice here anyway is to write code which compiles and works without these target features enabled

Without target features we are discarding a lot of the work that went into making CPUs faster over the past 20 years and programs will utilise only a fraction of the available computing power. Of course it would be nice if authors also write fallback code, but that is for the authors to decide. Not all crates compile/work on every OS either.

[jethrogb]

hen one of the dependencies requires a feature that was not selected by the top-level project, Cargo should refuse to build.

In the mean time, those dependencies could do something like:

#[cfg(not(target_feature="avx"))] const E:MUST_BE_COMPILED_WITH_AVX=();

[alexcrichton]

Hm, true, it is indeed in the project section! I think for those purposes, however, it may be more appropriately configured in [profile.foo] as that’s already where configuration like opt-level and debuginfo are located.

But authors might want to write code that depends on a certain target feature, and if they do not provide fallback code, that target feature is definitely required — there is no getting around that.

Yeah it’s definitely true that Cargo needs to support this in some functionality, but there’s a question of influence here to. For example if this requires RUSTFLAGS, then that’s possible, but Cargo is still nudging towards doing the “proper thing” by adding a fallback. In that sense it’s largely just a question of how ergonomic this should be.

Without target features we are discarding a lot of the work that went into making CPUs faster over the past 20 years and programs will utilise only a fraction of the available computing power.

I find this a pretty interesting question as well, actually. I’ve been curious in the past if there’s a select few situations where we could do something like -C target-cpu=native or something like that to take advantage of the host CPU.

[ruudva]

it may be more appropriately configured in [profile.foo] as that’s already where configuration like opt-level and debuginfo are located.

I agree. My only concern with this is that you’d have to duplicate the features for every profile. (See also the first point under unresolved questions.) Or is there actually something like an “all” profile?

For example if this requires RUSTFLAGS, then that’s possible, but Cargo is still nudging towards doing the “proper thing” by adding a fallback.

I see your point. Crates could still provide a specialised implementation behind #[cfg(target_feature="...")], optionally one that falls back to the regular version after a runtime check. Or authors will just add a fallback that panics, or one that is a compile error as suggested by @jethrogb. Is there something we can do to encourage adding a fallback, as opposed to discouraging using target features at all?

A slightly related issue: if you went to the trouble of implementing specialised functions that use target features, but Cargo then discourages using them, or the default is to use the fallback version, that would be a shame. Especially if you do support runtime feature detection, so there is no harm in enabling the target feature. So you add “put this snippet in your .cargo/config to use the fast version” to the readme, and everything that depends on that crate now needs to add that to their readme as well …

I’ve been curious in the past if there’s a select few situations where we could do something like -C target-cpu=native or something like that to take advantage of the host CPU.

In my experience with C++, just doing that doesn’t magically make the code faster. Perhaps LLVM can vectorize a few loops or use a more efficient instruction here and there, but the real win comes from being able to do 4/8/16 operations at the cost of one, and this often requires invasive changes to the code. (Actually, my experience is that target-cpu=native produces a binary that crashes with an illegal instruction, but that is likely an LLVM bug )

[Amanieu]

I’ve thought about writing an RFC for something similar, however my approach is different. Rather than using an attribute in Cargo.toml, the code for a library would use a compile-time assert to ensure that any features it requires are available.

Example:

#![cfg_assert(target_feature = "avx", "This library requires AVX support")]

This can also be used for OS or architecture-specific code:

#![cfg_assert(target_os = "linux", "This library only works on Linux")]

#![cfg_assert(any(target_arch = "x86", target_arch = "x86_64"), "This library only works on x86")]

I don’t think it’s Cargo’s job to figure out exactly what features a library requires. With my approach any incompatible features (as defined by the top level project) will simply result in a compilation error when building dependencies along with a nice error message from the library author.

[ruudva]

This is really neat idea! Then the required-target-features field can be removed entirely.

[alexcrichton]

@Amanieu that’s a fascinating idea! It somewhat plays into the idea of static assertions as well, but it seems like a great way at least to me to indicate that a library only works on some platforms or with some features.

@ruudva yeah unfortunately there’s no [profile.all] “profile” for this, and in general I’ve felt a little uncomfortable with profiles recently (for reasons like this). I do also wonder if the “speed bump” that Cargo would provide of requiring RUSTFLAGS is just too much of a speed bump to promote usage of these sorts of features throughout the ecosystem.

One vector to take here is to greatly improve the runtime detection story here. One problem today is that even if you do runtime detection, there’s no way to force a function to be compiled with AVX support unless the entire crate has support. In theory your runtime detection would delegate to a number of differently compiled functions, but the story there in Rust is pretty bad.

I guess all in all my feelings are:

• If we can not worry about required-features in favor of something like @Amanieu’s proposal, that sounds great!
• If we can put features in [profile.*] sections (like opt-level), then that meshes well with what we’ve got going today. It’s somewhat unergonomic, but it’s arguably unergonomic for doing something like “turn on debug assertions in all modes” as well, so a solution there would solve both problems.
• Eventually we can bolster the runtime-detection story, although we may not quite be there just yet.

[ruudva]

There is one more issue with supporting runtime feature detection: for the specialised code to be compiled, the target feature must be enabled. But when the target feature is enabled, LLVM is free to use those features everywhere. Or am I missing something? For instance, if you enable SSE then LLVM will be free to use SSE registers, even if there are no simd types or intrinsics in the code. So the final executable will still not run on every machine.

So to properly support runtime feature detection, more fine grained control would be needed in the compiler. It seems that there is no easy way to do this with Clang or GCC either, except for setting the target features per translation unit.

[alexcrichton]

I believe more recent versions of LLVM support code generation attributes per-function so we can tell LLVM to enable SSE for just one function but not the entire program. There’s currently no way in Rust itself to specify this, however.

[eternaleye]

If you plan on exporting very short functions that do this, it would be a good idea to cache the relevant feature flag in a static mut AtomicUsize, since CPUID is a serializing instruction.

I’ll just leave this here:

<plug type="shameless"> https://crates.io/crates/ifunky </plug>

[bluss]

I filed this issue about target-cpu=native, which is a bit different (simpler!?) because it’s about opportunistic exploitation of the target’s features, i.e. code that does not require but greatly benefits from using non-default (rustc on x86-64) features like SSE4.2, AVX, etc.

[ruudva]

If you support target-cpu=native, is there any harm in supporting all target CPUs?

[Cyberax]

How about using ifuncs on Linux to select the best implementation at runtime?

[ruudva]

Another issue that I ran into with RUSTFLAGS: if you want to build for a target CPU with different features than your host CPU, build scripts get built with RUSTFLAGS too, which means they might not run on your system and you can’t build. So if the flags can be set from Cargo.toml, they should not be applied to build scripts.

[gnzlbg]

@Amanieu

FWIW you probably already know this but @ruudva might now. Right now you can do:

#[cfg(not(target_feature = "avx"))]
compile_error!("This library assumes availability of AVX and must be compiled with -C target-feature=+avx");

For everything else I pretty much agree with what @alexcrichton says.

If the problem is that RUSTFLAGS are not easily discoverable, or not very ergonomic, we could think about adding options to cargo that allow you to pass features, like cargo build --target-features=avx or similar, that get shown by the cargo build --help. Or maybe cargo build --help should just tell you about RUSTFLAGS and about how to query the available RUSTFLAGS.