Getting explicit SIMD on stable Rust

[newpavlov]

typenum is a nice hack which somewhat eases the pain, but it has it’s own limits and costs, e.g. for non trivial stuff type signatures became bloated, errors usually hard to read because types defined recursively (this can be fixed but still) and it has compatibility issues with arrays.

I think @Aurora implied something like this RFC.

[gnzlbg]

@burntsushi any progress on the pre-RFC ? Is there anything I could do to help?

[gnzlbg]

FWIW i’ve started testing the assembly generated by rustc on different architectures (x86_64, armv7, and armv8) in my bitintr crate (see check_asm.py and the asm/ directory). I’ve found a couple of issues already (in particular with respect to LLVM armv8 missing optimizations). If anybody working on SIMD support wants to use this maybe we could refactor it into its own cargo-script.

[rkruppe]

Then the shuffle from above could be written as let v2 = v1.xyz;, which to me is clearer and shorter. If we use the OpenCL syntax then it covers up to 16 elements (s0 - sF), which is enough for AVX512. These shuffles are completely portable and allows you to use the same code on ARM and AMD64 for example, which is very nice.

16 elements is enough for vectors with 32-bit elements, but not for smaller element types. Shuffles for those are not part of AVX-512 proper, but are included by further instruction set extensions. Besides, current hardware SIMD width should not dictate the design of vector types, nor should syntax be designed under the assumption that hardware won’t change in the future.

I do admit that dedicated syntax is quite convenient when you need to shuffle a vector, but for the above reasons it can’t be the only way to shuffle. If it’s even a good fit for Rust to begin with, which I’m not sure of.

But it has a catch. Since we’re not explicit with the types anymore, the compiler needs to be able to figure out the return type of that accessor. If we allow more than one f32x3 kind of type, then the compiler won’t be able to pick which one. This is where struct X becomes an issue. If we instead have vectors mirror [T; n], we can easily generate those types in the compiler without knowing the name of a specific struct.

This is true, but it points to an issue with wrapper types: The primitive type would be privileged, gaining special syntax that a newtype can’t emulate. If you wanted to add a method, or present a different/more expansive interface to SIMD, you’d need an ugly extension trait or client code would need to regularly take out the structural type, perform an operation (e.g., shuffle) with it, and then wrap it back up. Moreover, if you wanted to represent a vector with some invariant as newtype, you’d be out of luck.

So to encourage a strong ecosystem around SIMD computations, I believe our representation of shuffles should be expressible in library code to enable newtypes. Naturally, we don’t currently have the means to express a shuffle signature — but luckily, we don’t need fully generic shuffles right now, if we start with intrinsics. If, however, we start out with special syntax for shuffles, it becomes much harder to turn that into something that can be supported by libraries.

[Aurora]

This is true, but it points to an issue with wrapper types: The primitive type would be privileged, gaining special syntax that a newtype can’t emulate. If you wanted to add a method, or present a different/more expansive interface to SIMD, you’d need an ugly extension trait or client code would need to regularly take out the structural type, perform an operation (e.g., shuffle) with it, and then wrap it back up. Moreover, if you wanted to represent a vector with some invariant as newtype, you’d be out of luck.

I have never felt that this is a big issue in C++/ObjC, but I would love if there was some sort of macros(?) to provide libraries the ability to provide custom accessors on newtypes.

So to encourage a strong ecosystem around SIMD computations, I believe our representation of shuffles should be expressible in library code to enable newtypes. Naturally, we don’t currently have the means to express a shuffle signature — but luckily, we don’t need fully generic shuffles right now, if we start with intrinsics. If, however, we start out with special syntax for shuffles, it becomes much harder to turn that into something that can be supported by libraries.

I want a generic interface for portable code (target at least AArch64 + AMD64), and you want close-to-metal performance for a single architecture. Fortunately these goals are not blocking each other in any way, but they are unfortunately also pretty much orthogonal. I think that’s why we have different priorities and see things differently.

I realise that AMD64 is the platform that 99% of all Rust code targets right now, so maybe the right choice is to just ignore portability for now but I hope it is not.

16 elements is enough for vectors with 32-bit elements, but not for smaller element types. Shuffles for those are not part of AVX-512 proper, but are included by further instruction set extensions.

The VBMI permutes are not an issue since they are already expressible in the Rust type system, fn vpermb(a: __mm512, index: __mm512) -> __mm512 for example.

[rkruppe]

The VBMI permutes are not an issue since they are already expressible in the Rust type system, fn vpermb(a: __mm512, index: __mm512) -> __mm512 for example.

By that standard, there is no need for a generic shuffle operation since all extant and future shuffle instructions are available as intrinsics. But in fact, a generic shuffle operation is highly desirable, and it would be extremely sad if it was restricted to an arbitrary subset of sensible SIMD types.

Consider: LLVM happily shuffles vectors larger than the largest register supported by the hardware.

[rkruppe]

I want a generic interface for portable code (target at least AArch64 + AMD64), and you want close-to-metal performance for a single architecture. Fortunately these goals are not blocking each other in any way, but they are unfortunately also pretty much orthogonal. I think that’s why we have different priorities and see things differently.

I do subscribe to the view that platform-specifc types and intrinsics are the only things that can reliably stabilize right now, but I’m also arguing with an eye towards the future, where higher-level, portable SIMD libraries flourish. These libraries, and platform-agnostic code written using these libraries, will have good reasons to want to create newtypes of vectors. Any special magic that gets added to SIMD types now will make such wrappers feel less first-class, and thus hurt that pretty intrinsic-free code.

[scottmcm]

Incidentally, couldn’t we just use plain array syntax for shuffles as well? E.g. let v2 = Simd([v1.0.0, v1.0.1, v1.0.2]) in the Simd<T> scenario (or ignore the Simd() and .0 noise othewise).

Both of the following functions produce shufflevector <4 x i32> %0, <4 x i32> undef, <4 x i32> <i32 1, i32 3, i32 0, i32 2> in release mode today, FWIW:

#[no_mangle]
pub fn rearrange_via_tuplestruct(a: Simd4<i32>) -> Simd4<i32> {
    Simd4(a.1,a.3,a.0,a.2)
}

#[no_mangle]
pub fn rearrange_via_array(a: Simd4<i32>) -> Simd4<i32> {
    [ a[1], a[3], a[0], a[2] ].into()
}

(Helpers, including horrible transmutes as a simd struct can't have an array in today's nightly)

#[repr(simd)]
pub struct Simd4<T>(T,T,T,T);

impl<T> std::ops::Deref for Simd4<T> {
    type Target = [T; 4];
    fn deref(&self) -> &Self::Target {
        unsafe { std::mem::transmute(self) }
    }
}

impl<T> From<[T;4]> for Simd4<T> where T : Copy {
    fn from(a: [T;4]) -> Self {
        unsafe { std::mem::transmute_copy(&a) }
    }
}

a.1 and a[1] are the same LLVM GEP, so struct vs tuple vs array doesn’t really matter to it.

Of course, as Aurora said, this is the simplest case in release mode, not a proof of bet-your-business-on-it-ility. But I’m not convinced that either is “very hard to read”; they seem to quite explicitly and directly show what’s happening.

[hsivonen]

It should in most ways act like a [T; n] array

Is that actually desirable from the point of view of making costs explicit in a systems language?

shufflevector

I thought exposing generic LLVM shuffling had been deemed out of scope for the initial release-channel SIMD feature, and initially only shuffles that map to specific instructions would be provided in a manner similar to C intrinsics even if underneath the API they generated shufflevector in the LLVM IR. Is this not so?

(Generic shufflevector is rather bad at making costs explicit. For example, LLVM uses shufflevector to view the higher or lower half of a NEON quadword register via its doubleword aliasing, but of course such viewing is only needed for type system purposes and there’s no instruction generated at all. OTOH, if you want a shuffle that the ISA doesn’t have a single instruction for, you get a non-obvious number of instructions.)

Anyway, I’d like to avoid slowdowns in getting SIMD to the release channel, so I’m worried about reopening the discussion regarding the types. Firefox is about to drop support for non-NEON ARM, so soon SIMD-in-release-channel-Rust will not only be SSE2-relevant for Firefox but NEON-relevant, too.

[Aurora]

But I’m not convinced that either is “very hard to read”

That’s because you’re using a much better syntax

I was discussing, Simd([a.0.0, a.0.1, a.0.2]), which I find very hard to read. Simd(a.1, a.3, a.0, a.2) as in your example isn’t nearly as hard to read for me.

[jethrogb]

Looks like AVX512 will be available on regular server platforms: https://cloudplatform.googleblog.com/2017/02/Google-Cloud-Platform-is-the-first-cloud-provider-to-offer-Intel-Skylake.html Rust will need to support it.

[nagisa]

So, I’ve tried to implement memcmp and memcpy with what Rust currently has and couldn’t get a way to express the case that matters for implementing both of those: memory operands for some of the instructions. For example in SSE there’s pcmpeqb which admits an unaligned memory operand. That’s the missing piece for implementing memcmp between two differently aligned pointers.

Intel intrinsics do not cover this use-case (they all take vectors as well), and it might be hard to make LLVM emit such an instruction as well (to do that we’d need at least a way to do unaligned vector loads from the memory (i.e. load %vecty %ptr, align 1)).

[burntsushi]

@nagisa Could you unpack that? I don’t really grok what you’re saying. Examples would be super helpful.

[burntsushi]

@nagisa Additionally, why doesn’t something like this satisfy your desire for unaligned vector loads? Or what about the _mm_loadu_si128 intrinsic from SSE2?

[nagisa]

Oh, I guess I was wrong and pcmpeqb does not permit an unaligned pointer, my bad.

Additionally, why doesn’t something like this1 satisfy your desire for unaligned vector loads?

Terrible codegen in debug mode. _mm_loadu_si128 may be okay but is specific to architecture (i.e. its better to just write assembly then)

[nagisa]

So, in response to proposal to use [T; n] for representing SIMD. That forces our hand to only ever support uniform vectors, whereas there may be systems (esp. in the future: see RISC-V) which support non-uniform vectors just fine.

Additionally, why doesn’t something like this satisfy your desire for unaligned vector loads?

So I tried using this and with optimisations LLVM fell back to using both unaligned loads and stores, rather than just unaligned loads, like I was intending. Explicitly crafting the IR to make LLVM translate correspondingly aligned loads and stores resulted in the intended instructions being emitted. That suggests that we might at least want to introduce some intrinsic to do arbitrary aligned loads and stores, where its possible to specify alignment independent of the data type (or we might eventually gain #[repr(align(N))] which would probably help with this).

[comex]

Note that there is already std::ptr::read_unaligned as of a few months ago (albeit not stable); the current implementation uses memcpy, but changing it to be intrinsic-based would be ‘just’ an implementation patch. That doesn’t address custom alignments other than 1, though.

[gnzlbg]

For those interested, there is a Final Comment Period ongoing to stabilize cfg_target_feature in this issue: https://github.com/rust-lang/rust/issues/29717

Since this has been discussed here a lot, I thought some of you might be interested.

[burntsushi]

Also, I’ve been slowly but steadily working on the SIMD RFC. I recently took a detour to prove that the simd crate could compile on top of our proposal: https://github.com/BurntSushi/simd/tree/stdsimd

[jethrogb]

Can we use this somehow? https://github.com/intelxed/xed/tree/master/datafiles