Would a transmute_lifetime() function be a good candidate for the standard library?

1

This isn't a serious proposal, I don't have time to carefully consider or write out anything, but I was curious if anybody else thought that a transmute_lifetime function might be useful to have in the standard library.

It's a simple enough function to write yourself, which is maybe an argument against having it in the standard library:

/// Helper to transmute a lifetime unsafely.
///
/// This is safer than just calling [`transmute`][std::mem::transmute]
/// because it can only transmute the lifetime, not the type of the
/// reference.
unsafe fn transmute_lt<'b, T>(v: &mut T) -> &'b mut T {
    std::mem::transmute(v)
}

At the same time, the standard library argues so strongly to use transmute only as a last resort, that it seems like there might be a place for a safer version of transmute that only transmutes lifetimes.

I've written this function just now for a helper in my own crate, and if you search transmute_lifetime on GitHub you can actually find at least a few matches for basically this function signature:

https://github.com/search?q=transmute_lifetime&ref=opensearch&type=code

One thought, though, would be to figure out if we could make it generic to mutable and immutable lifetimes, and maybe structs with only one lifetime argument. I'm not sure if that's possible.

2

For that, usually unsafe { &mut *(v as *mut _) } should work, too, right?

I’m more commonly interested (because, as far as I’m aware it really doesn’t exist) in the opposite, something that changes the reference target type while preserving the lifetime.

3 Likes
3

Yeah, that should work. Just my personal preference, I'd probably stick with an explicit helper function to make it easier to tell what transformation is being intended.

4

Another one that should work: unsafe { NonNull::from(v).as_mut() }. A bit less symbolic, but I don’t know if it’s really any better. Nope… that supports covariantly casting the target, as well as inputting an immutable reference.

5

I don't know that I've ever wanted to change the lifetime on a simple &mut like that, all the cases where I've done lifetime transmutes have been where the lifetime is somewhere deeper inside the type, e.g.

NonNull<task::Context<'static>>
&'a mut task::Context<'a>

unsafe fn(*mut (dyn Future<Output = T> + 'a))
unsafe fn(*mut (dyn Future<Output = T> + 'static))
2 Likes
6

Another way to write this is using std::ptr::from_mut:

unsafe { &mut *std::ptr::from_mut(mut_ref) }
2 Likes
7

My scenario so far has been a situation similar to implementing a mutable iterator over something like a vector.

I can use the vector's built-in get_mut() to get a mutable reference, but I'm not allowed to safely return that mutable reference from next() function directly, because it requires returning a borrow created in the function. The solution was to extend the lifetime of the &mut T borrow to that of the borrowed vector, asserting that I won't return multiple references to the same portion of that vector inside the iterator.

Yeah, I like that option, I think. Especially because it's documented as a safer alternative to a cast that may change the type without warning.

8

Generally… that’s exactly the kind of thing you don’t want to do! And I suppose discouraging this approach serves as a good reason not to have a transmute_lifetime function.

In Rust mutable access to a value means unique access to the whole value and all of its elements. If you have a &mut Vec<T>, use get_mut to get a &mut T, then use lifetime transmutations to re-access the &mut Vec<T> whilst keeping the first &mut T, the borrow checker doesn’t help prevent it anymore, but usage of &mut Vec<T> does assert unique access to the vector and all of its elements and may make your &mut T reference immediately invalid.

So using lifetime transmutation to access multiple (different) vector elements mutable is not the proper way to do it. In reality, the implementation details of Vec involving indirection through raw pointers means you don’t actually necessarily encounter true UB if you do this (edit: well, turns out at least as far as miri is concerned it is UB) and the language rules around how much unique access a mutable reference actually asserts through a layer of indirection aren’t entirely finalized yet. It still isn’t the proper way!

The proper way to access multiple elements of a Vec is as follows:

  • obtain a *mut T to the start of the vector’s buffer via .as_mut_ptr()
    • this kind of pointer is very different from the pointer you can get by casting &mut v[0] into a *mut T, the latter only has provenance over the first element, whereas the one from as_mut_ptr allows access to the whole vec
  • use pointer offset, and dereferencing to create (possibly multiple, non-overlapping) mutable references to the elements
    • i.e. &mut *ptr.add(ix)

This is (essentially) the way IterMut for slices is implemented. It is also similar to how the unstable get_many_[unchecker_]mut is implemented (click source here); though that one uses the get_unckecked_mut method for *mut [T] instead, whose implementation you can track back to a .add(ix) operation, too.

Same idea for split_as_mut which is also implemented by getting a *mut T to the start, doing .add(offset), and creating new sub-slices from the pointers.

A different approach that can also work for vectors/slices in particular is to create a slice of cells, &[Cell<T>], then index into that, and finally on the &Cell<T> reference to an element call Cell::as_ptr as in &mut *element.as_ptr().

TL;DR of these approaches: creating two &mut T references to elements, even distinct elements, is done, logically, by starting out with two live mutable ways to access the whole vector, and since the resulting &mut T’s should be live at the same time, the original accesses must have been aliasing (shared, mutable access). This thus requires one of the 2 ways in Rust to do shared mutability: either raw pointers[1], or something using UnsafeCell. Above solutions cover both of these approaches. Transmuting lifetimes would instead result in using ordinary, mutable references for the purpose of shared mutability, which is forbidden.

  1. The approach must then generally be: (1) create raw pointer to whole data structure, (2) do the splitting up of access to individual, non-overlapping components/items in “raw pointer land”, (3) create the mutable references that are live at the same time only to the non-overlapping components/items, (4) ensure, perhaps via wrapping in a safe API, that the original mutable reference that was turned into a raw pointer is not accessed anymore (and thus not active) all the way until all mutable reference to components/items are no longer in use; once the original mutable reference is used again, don’t use the raw pointer anymore that you had created ↩︎

6 Likes
9

There is one other way, which is less powerful but doesn't require unsafe: You can use pattern matching to destructure an &mut [T] into one or more &mut Ts to elements at the beginning or end and a new &mut [T] for the rest:

struct OutsideInIter<'a, T> {
    v: &'a mut [T],
}

impl<'a, T> Iterator for OutsideInIter<'a, T> {
    type Item = (&'a mut T, &'a mut T);
    fn next(&mut self) -> Option<Self::Item> {
        let [l, rest @ .., r] = std::mem::replace(&mut self.v, &mut []) else {
            return None;
        };
        self.v = rest;
        Some((l, r))
    }
}
1 Like
10

Ah, right, the compiler directly supports a few cases of borrow splitting, too, without the need to go through the intermediate raw pointer. So I should have qualified that this was discussing manual creation of split borrows, without the direct compiler support.[1] The other typical example the compiler supports - besides pattern matching - is partial borrowing, such as accessing both &mut x.foo and &mut x.bar fields of a struct, at the same time.

  1. Only in the absence of compiler-supported borrow splitting, it is true that creating two references to elements/things requires starting with two live ways (references/pointers) to access them. ↩︎

1 Like
11

Great explanation, thanks! And yeah, you're probably right, that's a good reason not to have a transmute_lifetime function.

I'll definitely beware of wrongly splitting mutable references now!

Incidentally, in order to reduce some bounds-type checks that were already done by the iterator, I ended up replacing all of my iterator implementations just yesterday with by using my own get_unchecked_mut() methods, which return raw-ish pointers. I'm using the PtrMut<'a>, type from the bevy_ptr crate, which is essentially a raw pointer with a lifetime.

Miri is super awesome, and I'm already using it for all my test cases, and it looks like things are good so far. :slight_smile:

12

I actually just now ended up again writing some code that wants to transmute UserType<'a> to UserType<'b>[1]. We do guarantee that such a transmute is layout-preserving [source], but transmute is uber-powerful; a weakened transmute_lifetimes which only permits lifetime-variant transmutes and forbids any others would make code using that kind of transmute easier to validate.

Notably, unlike the OP function which is limited to changing the top-level lifetime on just &mut _, this version isn't implementable in user code, and would require compiler support.

Side note, it sounds like what you want to use is actually ThinSlicePtr... though there's only a &[T] version currently in the crate, whereas you'd want &mut [T].

  1. More specifically, I'm repeatedly smuggling covariant UserType<'a> as UserType<'static> (a lifetime lying pattern I've called 'unsafe before) through a channel in order to emulate MCP-49 style semicoroutines on top of futures and bevy's system parameter machinery. ↩︎

3 Likes
13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.