Pre-RFC: a vision for platform/architecture/configuration-specific APIs

[josh]

Given that you can’t currently tell rustc “throw in some avx instructions when compiling this function” I don’t really see the value of this proposal. You could force use of those instructions in an asm!(…) block, but if you’re doing that, you might as well manually insert the feature detection there, and handle the ‘feature unsupported’ case. And since you’re now handling the ‘no avx’ case, you don’t need to declare your function #[cfg(cpu(avx))] and all is well.

I had asm! in mind when I wrote that. And no, not every asm! block should do its own autodetection; sometimes you want a function that assumes the CPU feature set, so you can call it from another function that already knows the feature set and avoid duplicate detection code.

Perhaps it would be better to investigate doing this kind of stuff automatically inside LLVM (perhaps such a thing is possible already? I wouldn’t know), e.g. by teaching it to recognize an attribute that informs it to always compile with some cpu-features forced enabled

GCC has a mechanism like this; see __attribute__((target(...))) and __attribute__((target_clones(...))), for instance. The former allows you to optimize a function specifically for the target architecture; the latter constructs several functions and an ifunc resolver to choose between them at runtime. (The latter has some overhead that you may not always want, though.)

It would need to be just a warning by default for backwards compatibility.

Wouldn’t it only apply if you call an appropriately tagged function, which didn’t exist to call at all in a prior version? How could that break backward compatibility?

This boils down to inline assembly we can’t verify anyways

Not necessarily just inline assembly. Consider intrinsics, for instance. My motivation to bring this up came from a GCC bug, where it didn’t allow calling a function optimized with #pragma GCC target("bmi2") (such as the intrinsics in <bmi2intrin.h>) from a function not compiled for that target, even though the generic function in question called a cpuid intrinsic first. No inline assembly involved, just compiler intrinsics.

[Thiez]

I had asm! in mind when I wrote that. And no, not every asm! block should do its own autodetection; sometimes you want a function that assumes the CPU feature set, so you can call it from another function that already knows the feature set and avoid duplicate detection code.

I imagine privacy can take care of that without attributes. If you ensure that the pub functions in a module do the feature detection, then the non-pub functions can avoid the detection code.

[josh]

A module full of functions for a specific target instruction set may want to mark its pub functions accordingly, and leave out the detection.

In any case, I don’t think that needs a new mechanism; rather, I just want to make sure the mechanism proposed here doesn’t completely omit any possibility of telling the compiler “yes, I really do mean to call a more-specific function from this more-general function”.

[oli-obk]

There’s one major problem with the lints. They run so late, that they never see a function that has a cfg attribute which causes the function to disappear. We’d need a new kind of lint that runs on the pure parsed AST (so before this line)

[Ericson2314]

Oh it’s worse than that. It’s infeasible to brute force all possible configurations individually, so I’m pretty sure we’d need to somehow do name resolution for them all at once.

[nikomatsakis]

As an example, the core float case could optionally “assume a float-capable scenario”, enabled by the float feature, which is in its default features. A kernel (which is a binary crate) would omit float from its scenario bounding set, and thus if a dependency pulled in a float-enabled core, it would report an error.

Hmm. This feels related to the current “require-provide” stuff around allocators and panic runtimes.

[eternaleye]

Yeah, agreed - there’s a very similar shape to the way those crosscut the crates involved in a resolution.

EDIT: Hm, although there is one divergence worth noting - allocators and panic runtimes are “select one”, while scenarios as described are additive (much like cargo features) - in essence, scenarios as I framed them are “cargo features for the runtime environment”, plus the ability to restrict which scenarios are permissible.

[gnzlbg]

The only issue I see with this is “who controls these lints”? Do we have to wait for a new version of the compiler to appear? Or can external crates add their own?

For example I would like to add to the list, next to SIMD, support for Bit Manipulation Instruction Sets, that is, support for Intel’s BMI, BMI2, and AMD’s TBM instruction sets.

The motivation is that right now it is hard to write a fast quad-tree or octree in Rust on Haswell because any octree using BMI2’s parallel bit deposit/extract instructions is going to beat it by 2-4x when computing Morton codes. If one uses the llvm-intrinsics crate for this the lack of a cfg macro makes it hard to provide a fall-back for other architectures.

The only way I could workaround this right now is to modify the compiler myself, submit pull-request, and wait for this to get accepted and stabilize.

While this should happen anyways, there are a lot of scenareos in real-life, one cannot always expect the compiler to know about them all, nor expect the compiler devs to care about them all. I basically gave up on bit manipulation algorithms for num because once the API was more or less stable I tried to make them fast and this turned out to be impossible without using intrinsics, and the support for the bit manipulation intrinsics was and still is zero.

So I guess my question is, how will external crates be able to easily add their own constraints to the mix? That is, how can my bit manipulation algorithm library add a new architecture/instruction set like BMI2 and use a cfg flag on that for providing fallbacks for the cases in which this instruction set is not available?

[retep998]

If we do add this sort of complex configuration lint, I’d really appreciate if it was optional because I can imagine there’d be a lot of edge cases that result in that lint taking a horrifying amount of time as it tries to resolve every cfg combination. I’ve been bitten enough by compile time regressions as is.

[brson]

Another category of feature that I believe falls into this same design space is the inclusion of the allocator and unwind crates. These have a global affect on all downstream components - you are either in the ‘jemalloc allocator scenario’ or the ‘system allocator scenario’; the ‘panic unwind’ scenario etc.

I don’t think it makes sense to tack this onto the lint infrastructure - they don’t seem particularly related to me. These ‘scenarios’ comprise a set of constraints that must be obeyed by all downstream crates. I like the ‘scenario’ terminology - we should consider it a fresh subsystem and not part of lints. Every time a crate introduces or depends on one of these global features or configurations it introduces a new constraint to the compilation scenario.

[eternaleye]

Another category of feature that I believe falls into this same design space is the inclusion of the allocator and unwind crates. These have a global affect on all downstream components - you are either in the ‘jemalloc allocator scenario’ or the ‘system allocator scenario’; the ‘panic unwind’ scenario etc.

As I said in my earlier post, those crosscut in a similar manner, but have some important differences - for one, they’re mutually exclusive. One can’t be in the “panic unwind + panic abort” scenario, while “atomic + float + linux” makes perfect sense.

A possible way of reconciling these would be to allow one provider for each scenario - thus, crates can rely on being in a “allocator” scenario, and exactly-one crate could be (say) “=allocator”, enabling that scenario.

The top-level crate, meanwhile would define a bounding set - say, “atomic + float + linux + alloc=jemalloc”. This would permit any single provider for each of atomic, float, and linux, and mandate that if anything relies on alloc, that MUST be satisfied using the jemalloc crate.

I think this would actually permit full user-level creation of scenarios outside the stdlib, which is quite nice.

[Ericson2314]

@eternaleye well I think we’d want to have and/or so yes “panic unwind & panic abort” makes no sense but “os(windows) & os(linux)” doesn’t either.

---

Anybody read my first reply that this can be scenarios can be an exhaustiveness check on cfg, so we don’t need a new annotation? Unless I’m missing something this is a nice free simplification.

[eternaleye]

“os(windows) & os(linux)” doesn’t either.

Winelib on Linux would be “=windows”, to use my terminology, for one. Cygwin gives Windows + Unix, as does midipix.

And no, such an exhaustiveness requirement would be

1. A huge amount of boilerplate for each crate
2. Semantically backwards
3. Relatively inextensible

[Ericson2314]

winelib, Cygwin, midipix

Fair enough of the principle, but do any of those work like that? Last I checked there is a huge amount of code in std and the community that assumes a unix-windows dichotomy.

1. A huge amount of boilerplate for each crate

Huh? My proposal is that we cfg just like today, but something checks to ensure regardless of the (valid) combination of cfg tokens, there are no unresolved identifiers. The compiler, not the users “proves” that that the cfgs are total. If anything the problem would be compilation time bloat, not code bloat.

2. Semantically backwards

I am not sure what you mean.

3. Relatively inextensible

Do you mean any(os(windows), os(unix)) seems total, but then somebody adds multics to rustc, so the code wasn’t so portable after all? We can avoid that.

[dobenour]

Intrinsics support for bit operations is necessary.

I think that Rust should support all intrinsics that LLVM does.

[Ericson2314]

This may sound goofy, but I’d love to see a scenario where Rc is gone and destructors are guaranteed. Everybody gets to eat their own cupcake :).

[rkjnsn]

I looked at this during the “should forget be unsafe” discussion, and I do believe it should be possible to have destructors guaranteed while having reference counting. (Specifically, the guarantee would be that an object would be guaranteed to be destroyed by the end of it’s valid lifetime.) Rcs containing only 'static data could be used exactly as today, since they would be okay to leak. (Their valid lifetime is “forever”.) Rcs containing non-'static data would either be statically prevented from containing cycles (which would require a way to specify that one lifetime strictly outlives another) or would be used with a scope-guard cycle collector that makes sure any cycles are cleaned up.

[ubsan]

I’m not a big fan of this. It feels “worse than C”. At least in C or C++, you need to #include <platform_specific.h> before using any platform specific APIs. I would rather have any platform specific APIs stuck in some sort of std::sys, which says “if you use these, your code is not guaranteed to compile everywhere”, kind of a thing… I’m not sure, though.

I want Rust code to be, by default, portable.

[alexcrichton]

I’ve posted a follow-up thread to this where I hope to flesh out the concept of scenarios to be a bit more concrete. Feedback welcome!

[aturon]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.