Yeah, that is correct.
The concern is with code that has a pointer x: *const usize that points to an array with a mix of pointers and usize and uses x.add(N) to access the n-th element. That will lead to complete nonsense on CHERI, it doesn't have to crash cleanly.
By inefficiency, are you referring to collections of indices like e.g. Vec<usize>?
I'm not terribly concerned about that, given that usize is already awkwardly large on 64-bit targets. I'm all too familiar with the dance of refactoring my Vec<usize> fields into Vec<u32>, so if I ever needed to target CHERI not much would change in this regard.
The inefficiency of using something uintptr_t-sized for size_t purposes generally, yes.
Which is discussed in the thread and linked discussions above, with some finding it acceptable if lamentable and others finding it unacceptable. Please read the thread and linked discussions in lieu of rehashing arguments which have already been made.
It's not just the size footprint. Arithmetic too. If code is written to assume that doing bit-twiddling and integer math on usize is cheap then it could get bitten hard on a platform where usize is not the native register width.
In a lot of places we even do things like mem::size_of::<usize>() * 2 or such to say "two register widths" (llvm's scalarpair) or * 4 for "about the right size to SIMD it".
To be clear, is this another perf regression or could it also be a correctness regression?
If there's a correctness argument for usize being register-sized on a CHERI target that would be a interesting new contribution to this discussion.
I was only talking about perf.
Great, this is the reason I asked, I have been following this thread and its sister thread for a few months now but haven't seen anyone define "efficient" yet, and I suspect people may be meaning entirely different things by it.
For collections of indices, I don't think it is disputed that a larger usize would be (memory) inefficient.
For arithmetic, I would be interested in benchmarks on this. It isn't obvious to me how much code in the wild is doing arithmetic on indices or how sensitive this is to optimization passes.
For things like scalarpair/simd, I'd similarly be interested in benchmarks, and I'd note that we don't promise that tricks like e.g. [usize; N] will get recognized as SIMD-able. I've had rustc fail to recognize my hints that something is SIMD-able simply because I introduced or removed a branch in some (seemingly) unrelated code.
TBH, even on non-CHERI, I often wish we had an index type where usize: From<index> and isize: From<index>, because the only place where indices larger than isize::MAX are ever needed is on ZSTs, and I'm unconvinced that they're ever useful there. And having the range restriction would make Option<index> nicer, would mean that LLVM would know that (i + j)/2 doesn't need to worry about wrapping, that a[i..i+4] can be implemented in a way that doesn't need to worry about wrapping, etc.
Unfortunately adding that is hard, since it needs inference features that don't currently exist.
There is also qemu as an option. But the barrier to entry seems a bit high. Most of the instructions I found for this involve building the CHERI fork of qemu yourself. I did find one page mentioning a Docker container that apparently has the necessary qemu binaries and FreeBSD image. But even then, it's inconvenient to have to emulate a full system. Would be nicer if there was support for qemu's userland emulation mode, but I don't see any mention of it being supported. Maybe that will happen when CHERI Linux is in a more mature state?
Anyone who's actually involved in CHERI, feel free to correct me on the above. 
In any case, it sounds like Miri support would be quite helpful.
I'd expect anything looping over views into n-dimensional arrays doing lots of adds and muls (including saturating variants or looking at the status flags) and maybe the occasional div/mod to calculate the flatted index. Even basic array chunking or windowing involves a those. And it's not just the overhead of the machine instructions but also making loop analysis more difficult for the optimizer since it now also has to deal with additional truncations or checked conversions from 128bit to 64bit indices.
I have not done any benchmarks though.
I think 128bit usize is also just conceptually wrong. The extra bits don't actually get you anything when doing ptr2int casts, they are only ever a burden. And there's still a risk of subtle UB when using C bindings that assume usize is size_t. So the entire discussion around "do we expect crate authors to do the work of making their crates compatible with CHERI" (answer: no, we do not) exists for either choice of usize. And between the two, a 64bit usize is IMO clearly preferable.
Fundamentally I don't think there is a path to CHERI in Rust without a risk of subtle UB. But I think that's okay, porting to CHERI needs careful auditing and testing; the CHERI people are used to that from porting existing C code.
Won't the exact same issue appear for C API that expect intptr_t == usize. Though I believe such APIs are quite a bit rarer. Which would indicate that regardless of what we make usize, the other option still needs to be added if you want to allow portable bindings.
You could maybe argue for type punning pointers on the Rust side to intptr_t instead. But I seem to remember that there is some weird architecture (not currently supported by rust, m68k maybe?) where the calling convention differs for pointers and integers. What with ongoing work for gcc backend/reimplementation that may soon become a problem rust has to tackle as well.
I think it is inevitable as rust gains popularity that it will get ported to stranger and stranger architectures. Either by embedded people / those stuck on weird systems for whatever reason, or by hobbyists (out of love/spite/for the heck of it). Might as well bite the bullet now and come up with a general and flexible answer so we don't have to repeat this discussion in two years time.
And it is okay that every crate won't work on every architecture. It is not like every crate works on every OS either. It should be on those who want a crate to support something unusual to put in the work and make the patches and engage with upstream in good faith.
m68k is supported by rust: m68k-unknown-linux-gnu - The rustc book
Well, I was wrong then, it is an even more immediate problem. A quick Google indicates that m68k has separate data and address registers.
But it seems we really do need (to properly support existing all platforms) to treat pointers and integers separately. Type punning between rust pointers and intptr_t won't cut it. And for CHERI type punning intptr_t to a rust pointer would probably work, but usize needs to match size_t.
The only way to reconcile these statements seem to be to introduce (at least for FFI purposes) separate types for intptr_t and size_t in Rust. Perhaps the recommendation should be to only use them for FFI (std::ffi::c_intptr)? But I don't see a future where rust can fully keep pretending that these are all the same thing.
That said it doesn't need to be something that most programmers ever have to deal with. Just those work with FFI bindings or other low level things. And the compiler will catch mistakes (assuming bingen etc gets it right) since if c_intptr is u128, implicit casts to other types will error.
Edit: reference: m68k - Lazarus wiki
The nice thing about CHERI is that we don't need to go all the way to arbitrary size_t / uintptr_t combinations. After all, a pointer stored in a uintptr_t is useless on CHERI anyway (once you start doing any integer things with it).
So I wouldn't mix that up with other discussions like m68k. (I also don't see the problem with m68k; as long as the compiler generates correct code for int2ptr/ptr2int, it shouldn't matter that there are different registers. But again, it seems off-topic to discuss m68k in a CHERI thread. This is not a "let's support all weird platforms" thread.)
I don't believe it is off topic to ask how to write a rust FFI signature that is potable across existing platforms rust support while also supporting this new fancy architecture. That seems extremely on topic to me. Let's make a concrete example. How do I write the correct extern block for this C signature:
void myfunc(void* a, intptr_t b, size_t c);
Now, all I'm asking for is how to write the corresponding extern in Rust, such that it is portable across all existing architectures that Rust support as well as CHERI. If that is off topic I don't know what is on topic.
Note that CHERI intptr_t is not __int128 — the integer (arithmetic) range is the same as vaddr_t (u64) despite the increased alignment and storage requirements. Additionally, AIUI pointers are actually 129 bits and ever placing a pointer in a non-pointer type will result in an invalid pointer (strict provenance). intptr_t carries this provenance, __int128 does not.
- intptr_t, uintptr_t
These integer types should be used to hold values that may be valid pointers if cast back to a pointer type. When an
intptr_tis assigned an integer value – e.g., due to constant initialization to an integer in the source – and the result is cast to a pointer type, the pointer will be invalid and hence non-dereferenceable. These types will be used in two cases: (1) Where there is uncertainty as to whether the value to be held will be an integer or a pointer – e.g., for an opaque argument to a callback function; or (2) Where it is more convenient to place a pointer value in an integer type for the purposes of arithmetic (which takes place on the capability’s address and in units of bytes, as if the pointer had been cast tochar *).The observable, integer range of a
uintptr_tis the same as that of avaddr_t(orptrdiff_tforintptr_t), despite the increased alignment and storage requirements.
TL;DR: you don't. Current Rust says that usize is both intptr_t and size_t. Targets where this isn't true are an API break, which is the point of this thread. There are unstable type aliases for core::ffi::c_size_t and core::ffi::c_ptrdiff_t but no such aliases for intptr_t yet. (And arguably these aliases should be the bit sized types and not the _size types, tbh.)
The most correct translation of that function signature is to not use integers on the Rust side for anything that can carry provenance. Pray that your target doesn't give void*C and intptr_tC different ABIs and use *mut c_voidRust for both. Or if the API Crimes are absolutely necessary, use something like ::sptr::iptr.
Fair enough, but today it works if you map both integer types to usize, for all currently supported targets. CHERI breaks a this, so if Rust wants to support CHERI we need new types to express this with. I think that is demonstrated as unavoidable, yet part of this discussion seems to be about "well, do we reaaaly need to?". This is confusing to me when such a simple counter example show it.
And I don't think (since m68k is a supported target, albeit a tier 3 one) that "just map to pointers" is the right answer. Maybe m68k is such a niche target that it really doesn't matter (it is a legacy architecture as far as I know). But then, should rust even attempt to support it in the first place, if it is just lip service anyway?
Huh? How does that work, where is that additional bit stored, taking into account padding etc? And I presume you still use normal RAM with this thing (not special ram with some extra hidden away bit like ecc)? As far as I know DDR5 memory etc has natural alignment for loads/stores as well. The only way I could see it working was if every pointer was actually 256 bits, or the extra bit is stored out of band in a bit map somewhere else in memory. Neither seems particularly performant.
It moves, depending on whether the pointer is in DRAM or in cache.
There's a block of DRAM reserved for validity bits, where each bit in the DRAM tells you if the corresponding 128 bit pointer in the rest of DRAM is a valid CHERI capability or not.
Once that pointer is loaded into a cache line, the validity bit is attached to the cache line, expanding (e.g.) a 64 byte (512 bit) cache line by an extra 4 bits for validity information.
Fair, I misunderstood your point. I didn't realize it is entirely about FFI signatures, sorry for that.
That summary is a bit too brief -- the 129th bit is a magic metadata bit that is only carried along under specific conditions and doesn't show up in addressable memory.
My interpretation was that the magic bit is just always preserved when you do a 128bit load from memory that happens to exactly be one pointer, which would mean __int128 loads would also preserve the magic bit. But the CHERI people here can hopefully comment on that.