Is this program intentionally rejected?

[mcy]

The following does not compile on stable, much to my surprise:

fn foo(ptr: &impl Tr) {
  // For extending the lifetime of a reference you can 
  // *guarantee* will not escape the function it is extended in.
  unsafe fn extend<'a, T>(t: &T) -> &'a T {
    std::mem::transmute(t)
  }
  let _ = extend::<'static, _>(ptr);
} 

Transmute (or perhaps the borrowck) appears to be unwilling to admit a &'static T if I don’t assert that T: 'static; this can be fixed by writing impl Copy + 'static. I don’t want to require this because my uses of this function are for something along the lines of

fn foo(ptr: &(impl Tr + Send)) {
  // Extend this reference in order to make the borrow
  // checker not attach a lifetime to the closure below.
  // This is safe, because those closures are all destroyed by
  // the join at the end of this function.
  let ptr = extend::<'static, _>(ptr);
  for .. {
    thread_pool.exec(move || { ptr.bar(); ... });
  }
  // Ensure no extended references escape this function.
  thread_pool.join(); 
}

There’s a workaround with dyn Tr + Send, but I’d like to know if this is an intentionally rejected program. If it is, this seems like a really weird place to take the edge off transmute.

[vorner]

I believe this is rejected on purpose by the borrow checker (transmute is not special there in any way). Imagine your T = &usize, then you would be creating things like &'static &'a usize which is, um, bad… so that thing it asks you to do is to say that you’re not going to extend the lifetime past some other references/lifetimes hidden in T. These things just Must Not Exist.

Anyway, the recommended way to extend a lifetime in somewhat saner way than transmute is cast the reference to raw pointer and back.

[mcy]

@vorner

Imagine your T = &usize , then you would be creating things like &'static &'a usize which is, um, bad… so that thing it asks you to do is to say that you’re not going to extend the lifetime past some other references/lifetimes hidden in T.

In fact, I'm aware of this. However, I can prove that no access happens beyond the region 'a, though I can't prove this to the compiler. While this is a type that should never escape into safe Rust, it's a bit frustrating that I can't tell the compiler "I know that this lifetime bound in this method should be shorter because of this join point later on."

@vorner

Anyway, the recommended way to extend a lifetime in somewhat saner way than transmute is cast the reference to raw pointer and back.

I tried this before resorting to transmute; it didn't work, since, as you pointed out, it's the borrowck being upset.

[joeschman]

Do you maybe want something like this:

fn foo<A: 'static>(ptr: &A) {
  unsafe fn extend<'a, 'b, T>(t: &'a T) -> &'b T {
    std::mem::transmute(t)
  }
  let _: &'static A = unsafe {extend(ptr)};
} 

[cuviper]

@mcy

Anyway, the recommended way to extend a lifetime in somewhat saner way than transmute is cast the reference to raw pointer and back.

I tried this before resorting to transmute; it didn’t work, since, as you pointed out, it’s the borrowck being upset.

I believe the point is that instead of creating this "longer-lived but not really..." reference, you should just pass around the raw pointer. This way you're on your own to ensure that dereferences only occur within an appropriate lifetime.

This is roughly what Rayon does, which appears to be a similar case to your thread pool. The rayon JobRef carries a raw pointer, even for scoped cases where the caller must not outlive a given lifetime.

[vorner]

@mcy

In fact, I’m aware of this. However, I can prove that no access happens beyond the region 'a , though I can’t prove this to the compiler. While this is a type that should never escape into safe Rust, it’s a bit frustrating that I can’t tell the compiler “I know that this lifetime bound in this method should be shorter because of this join point later on.”

No, the problem is somewhere else. What you would actually want to transmute into is not &'static &'a usize, but &'static &'static usize (but T can't be both at once). It's not about proving something about lifetimes, it's about the former is not even a type in some sense.

(And, by the way, is your code really safe even if it worked? What if the second call to thread_pool.exec panics? The .join won't be called, but the first thread will continue running)

[mcy]

@cuviper

you should just pass around the raw pointer.

Alas, this doesn't work; once inside the closure, the compiler can't prove that the 'closure lifetime is in any way related to the 'ref lifetime we started with, so I can't call the &self methods I need.

@vorner

No, the problem is somewhere else. What you would actually want to transmute into is not &'static &'a usize , but &'static &'static usize (but T can’t be both at once). It’s not about proving something about lifetimes, it’s about the former is not even a type in some sense.

Right; converting to &'static &'static usize is the morally "correct" thing to do, but unfortunately I'm using a generic parameter; there's no way for me to take a generic parameter and say "transmute this into the same type but with all lifetimes = 'static).

@vorner

(And, by the way, is your code really safe even if it worked? What if the second call to thread_pool.exec panics? The .join won’t be called, but the first thread will continue running)

I abort-on-panic all my programs, so for this example, we are assuming no panics.

[ExpHP]

One of the things I was wondering about when writing my blog series on “lock-based” borrow checking was: Why does &'a T require that T: 'a?

The tentative conclusion I came to was that it is not actually required for soundness, because the compiler should already be considering any locks held by T to be held by the borrow since it is covariant in T. And if the T is turned into a dyn type, then those locks will be represented in the + 'lt part of the dyn type. So as far as I can tell, the T: 'a requirement is nothing more than a lint, because it would be unsafe to release such types to arbitrary safe code (and you can easily use *const T instead).

---

P.S. This isn’t specific to transmute. You can’t construct a &'static &'a T through any means except in dead code (which is presumably allowed only because it generates no MIR?).

[mcy]

@ExpHP

And if the T is turned into a dyn type, then those locks will be represented in the + 'lt part of the dyn type.

Indeed, my workaround uses a dyn:

fn foo(ptr: &(impl Tr + Send)) {
  let ptr = extend::<'static, _>(ptr as &(dyn Tr + Send));
  for .. {
    thread_pool.exec(move || { ptr.bar(); ... });
  }
  thread_pool.join(); 
}

Though this strikes me as a bit strange, thinking about it... Why am I not required to write dyn Tr + 'static? Does the compiler take an axiom for<trait Tr: ObjectSafe> dyn Tr: 'static, or does my extend (regardless of implementation) silently attach a 'static (which seems... really wrong).

[ExpHP]

@mcy

Though this strikes me as a bit strange, thinking about it… Why am I not required to write dyn Tr + 'static ? Does the compiler take an axiom for<trait Tr: ObjectSafe> dyn Tr: 'static , or does my extend (regardless of implementation) silently attach a 'static (which seems… really wrong).

You're saying that this compiles? I cannot reproduce this. Can you reduce a full example?

[scalexm]

One of the things I was wondering about when writing my blog series on “lock-based” borrow checking was: Why does &'a T require that T: 'a ?

I'm not sure to understand, you are wondering whether e.g. &'static &'a i32 may be a well-formed type with no assumptions on 'a? In that case, I guess you would keep the reverse implication that the type &'static &'a i32 outlives 'static? Because with that reverse implication and if it were a well-formed type, it would be easy to craft a transmute in safe code without even constructing a value of type &'static &'a i32.

[comex]

Fun fact I discovered while playing around with this:

This code compiles, despite purportedly creating an &'static reference to a shorter-lifetime reference… but only with NLL enabled. Trying to do anything unsound with it seemingly still produces an error.

fn foo<'a>(r: &'a u32) -> &'static &'a u32 {
    Box::leak(Box::new(r))
}
fn bar(n: u32) {
    let huh = foo(&n);
    println!("{}", huh);
}

[mcy]

@ExpHP

You’re saying that this compiles? I cannot reproduce this. Can you reduce a full example?

Oh, yep, I used the following instead of extend:

std::mem::transmute::<_, &'static dyn T>(func)

I'm assuming that &'a dyn T is sugar for &'a (dyn T + 'a)... I guess? At least, the compiler inserting a 'static is the only way I can see this working.

[ExpHP]

@mcy

I’m assuming that &'a dyn T is sugar for &'a (dyn T + 'a) … I guess? At least, the compiler inserting a 'static is the only way I can see this working.

It works that way in function parameter lists, but I don't think it works that way in a function body (all elided lifetimes in a function body should be fresh inference variables). My best guess of what's happening here is that ::<_, &'static dyn T> has a + 'b lifetime parameter that defaults to 'static only because it is completely unconstrained. (but this is difficult to test)

After all, the signature of transmute offers absolutely no connection between the input type and the output type.

@comex

This code compiles, despite purportedly creating an &'static reference to a shorter-lifetime reference… but only with NLL enabled. Trying to do anything unsound with it seemingly still produces an error.

Right; you're allowed to create these types only in contexts where the specific value of 'a is not known (i.e. when it is a generic parameter). Once it is known, the compiler notices the discrepancy and rejects it.

...or something like that. In any case, this means it is possible to write functions that cannot actually be called, which certainly has bit me a number of times.

[scalexm]

Your foo function is a valid function, it’s just relying on implied bounds to infer that 'a: 'static. The fact that you can call it in bar with a non 'static lifetime is a bug:

Issue: Unsoundness related to implied bounds for return types

And as you can see, if you trick the compiler enough you can create unsoundness out of these non well-formed types (cc @ExpHP) . Try compiling in nightly (where the bug was fixed) and your code won’t work anymore.

[scalexm]

Rules about what the default lifetimes for trait objects are are stated in this RFC: https://rust-lang.github.io/rfcs/1156-adjust-default-object-bounds.html

Here the compiler indeed adds a 'static bound for you, which is the only valid lifetime anyway since you’re talking about a 'static reference to this type

[mcy]

@ExpHP

After all, the signature of transmute offers absolutely no connection between the input type and the output type.

Other than size_of::<T>() == size_of::<U>() magic, of course. =P

@scalexm

Rules about what the default lifetimes for trait objects are are stated in this RFC: https://rust-lang.github.io/rfcs/1156-adjust-default-object-bounds.html

Here the compiler indeed adds a 'static bound for you, which is the only valid lifetime anyway since you’re talking about a 'static reference to this type

Neat, thanks! I might go add the 'static to my code though, so it's less spooky... I do wish there was some equivalent way to hack the trait implementations of a type at compile time (i.e., transmuting from impl Trait<A> into impl Trait<B>... whatever that means) but honestly we can all agree that adding more sharp edges to transmute is a categorically Bad Idea.

I'm willing to pay for the dyn in this case. shrug