Specifying target-specific LLVM macros

I'm adding a new target to Rust that needs additional LLVM attributes to properly specify programs. Though not exactly my scenario, amdgpu_kernel and the OpenCL address spaces are similar to what I'm trying to do.

There are two approaches I can take:

  • Add a general-purpose #[llvm_attr(attr1, attr2)] that users can put on functions, arguments, etc and add wrapper macros outside the compiler for e.g. #[kernel] and #[global] that emit the correct llvm_attr. The nice thing about this approach is that it doesn't require me putting bespoke attributes for a single triple into rustc and this attribute might be useful for unrelated targets. Can deny attributes that rustc uses internally like noalias, readonly, etc.
  • Bake the e.g. #[kernel] and #[global] attributes into rustc that directly codegen the respective LLVM attributes.

I strongly prefer the first approach, though this has been shot down in the past.

Part of the challenge either way is that function parameter and local attributes quickly disappear after the AST gets lowered. So, you have to update the HIR, THIR, MIR, to propagate these attributes. With the second approach, you can use bitflags! to bake in the presence of the exact attributes you want, but with the first one it looks like you'd need to add an llvm_attrs field that points to an interned delimited list of attrs or something.

A spicy third option I guess would be to not do anything with rustc and do some proc_macro devil magic that emits C shims that specify these attributes and wraps the user program.

Thoughts?

The arguments in Make it easy to attach LLVM attributes to Rust functions · Issue #15180 · rust-lang/rust · GitHub still hold; we're not going to add an arbitrary passthrough for LLVM-specific attributes.

If you want to add support for address spaces, you should add dedicated attributes for address spaces.

(I can imagine two designs for that: an attribute for each address space, or an attribute for address spaces that takes an argument. The latter seems appealing, even though the supported address spaces will vary by target.)

What target are you looking to add?

1 Like

The target is actually kinda strange. It's not a real processor, but rather users are specifying fully-homomorphic encryption computations. Users need to specify which inputs are encrypted and which aren't. I'm imagining something like this

#[no_mangle]
#[fhe_program]
extern "C" fn mad(#[encryped] a: u64, b: u64, c: u64, #[encrypted] out: &mut u64) {
  *out = a * b + c;
}

where a, out are encrypted and b, c are plaintext.

4 Likes

FWIW, like it's possible but a permanently unstable internal feature to directly link to backend LLVM intrinsics from Rust source, I think it'd be reasonable enough to have a permanently unstable attribute able to attach arbitrary backend attributes to a function signature, as long as it's sufficiently communicated that it's a permanently unstable thing and any real usage should get a dedicated attribute independent of backend details.

Semantically it'd be similar to asm! — it does whatever it does and the developer is responsible for ensuring what it does is something allowed by the abstract machine. It'd also be beneficial in similar cases — the developer wants specific control over codegen and/or to experiment without needing to extend the compiler frontend yet.

But I can also understand t-lang's argument for not providing such an attribute.

2 Likes

Perhaps this falls under the first category, but I feel that "specific control over codegen" is a bit vague. It makes me think of things you could do in surface Rust (but would be for example slower). For example, media decoders, constant time crypto, etc.

But there cases of things you can't do in a high level language, and probably never will or should be able to. Various uses in kernels come to mind, such as messing with CPU control registers, flushing TLBs, syscall entry/exit paths, etc.

In my mind, "specific control over codegen" covers any case which is better described not in terms of the Rust Abstract Machine but instead what is done at a lower level, whether that be LLVM level annotation or assembly level properties. So to me it definitely includes those kernel use cases which bypass the AM and talk directly with the processor.

In many cases we want to find ways to communicate better what we're doing (e.g. extern "*-interrupt" is both more convenient and performant than writing a conservative global_asm! shim) so the compiler can understand and optimize, but providing a way to take specific control when desired has benefit. Like, even though it'd be painful to ever actually use, some kind of global_llvm! macro to inject fully arbitrary LLVM IR into a compilation module could be an awesome tool to have available alongside global_asm!.

3 Likes

Agreed. If it's #[rustc_llvm_attributes = "foo bar"] or something, as a way for people to experiment and demonstrate the value of potentially adding something that could be on a stabilization-track, that seems fine. (Like, as you said, how it's nice to have link_llvm_intrinsics on nightly even though we'll never stabilize it.)

But LLVM doesn't have a strong enough stability policy to ever have a stable passthrough to it, IMHO, since those things could easily break with newer LLVM, which Rust doesn't want under its stability policy.

5 Likes

If we put an allow-list in front of which attributes you can use, would that change any of the calculus?

For example amdgpu_flat_work_group_size(4, 5) could be allowed because it's orthogonal to Rust's codegen. naked and nonnull etc. would not.

LLVM could still remove or change the attribute in a future release, which isn't compatible with Rust stable's stability guarantees. If rustc is signing up to always translate some subset of attributes to whatever LLVM calls them in the future, then it should just be exposed as a native feature instead of as an instruction directly to the LLVM backend.

#[ffi_const] and #[ffi_pure] are perhaps an interesting comparison here. They currently exist just to translate the usage of __attribute__((const)) and __attribute__((pure)) in C headers, but it's starting to look like we'd rather spell these as #[pure(readonly)] and #[pure(nomem)] instead (mirroring the spelling and semantics used for asm!) to better match what the effect is for Rust instead of mixing abstraction layers between Rust's and the backend's semantics.

4 Likes

I see. I guess it also makes sense that if you're going to curate a list of attributes, you may as well lift them into more meaningful attributes such that you can maybe create equivalents in other codegen backends while providing cleaner syntax.

Hi, is there a github repository or somewhere else to follow the development of this?

Nothing yet public.