Hello and happy new year! I’ve made a draft RFC for the first time and would like to hear your feedback and suggestions. Thank you in advance!
- Feature Name:
owned_ref - Start Date: (fill me in with today’s date, YYYY-MM-DD)
- RFC PR: (leave this empty)
- Rust Issue: (leave this empty)
Summary
Introduce OwnedRef<'a, T : ?Sized + 'a> type in core. OwnedRef<'a, T> is a mutable reference &'a mut T where the callee is responsible for destructuring its contents, or a boxed pointer Box<T> where the callee is not responsible for deallocating the pointer. It’s almost the same as passing T by value, but different in that OwnedRef can contain dynamically sized types. Using OwnedRef, one can make by-value traits object-safe. Especially, Box<FnOnce> can work without intermediate FnBox trait.
Motivation
There has been a long-standing problem that we cannot call f: Box<FnOnce>, although we have the ownership of the closure. Several attempts have been made to solve the problem: especially rust-lang/rust#28796 and rust-lang/rfcs#998. Here I summarize the problem.
We can’t pass DST by value
The first cause of the problem is in the signature of FnOnce:
trait FnOnce<Args> {
type Output;
fn call_once(self, args: Args) -> Self::Output;
}
The point is that self has type Self, which is not necessarily Sized. Since we can’t pass unsized types by-value (but see rust-lang/rfcs#977 and rust-lang/rfcs#90 also), it prevents the following impls to compile:
impl<'a, A, R> FnOnce<A> for Box<FnOnce<A, Output=R> + 'a> {
type Output = R;
fn call_once(self, args: A) -> R {
<FnOnce::<A, Output=R> as FnOnce<A>>::call_once(*self) // *self is unsized
}
}
Box lives in the different crate than FnOnce
The problem looks easy if one can change the signature of FnOnce as follows:
trait FnOnce<Args> {
type Output;
fn call_once(self, args: Args) -> Self::Output;
// Similar to `call_once`, but `self` is passed in boxed form.
fn call_once_boxed(self: Box<Self>, args: Args) -> Self::Output;
}
However, we can’t do that because FnOnce and Box lives in different crates: FnOnce in core and Box in alloc.
The current workaround: FnBox
The current workaround goes this line: instead of injecting a new method to FnOnce, introduce another trait FnBox in alloc.
trait FnBox<Args> {
type Output;
fn call_box(self: Box<Self>, args: Args) -> Self::Output;
}
impl<A, F: FnOnce<A>> FnBox<A> for F { .. }
impl<'a, A, R> FnOnce<A> for Box<FnBox<A, Output=R> + 'a> { .. }
However, stabilization of FnBox is blocked by several issues. See rust-lang/rust#28796 for details.
Owned reference type: a missing piece
We can elegantly solve these problems by introducing the owned reference type &move T (syntax differ among proposals). It is similar to both &mut T and Box<T>.
- From
&mut Tside, an owned reference is a mutable reference where the callee is responsible for destructuring its contents. It has a lifetime as in&mut Tand can point to heap or stack. - From
Box<T>side, an owned reference is a boxed pointer where the callee is not responsible for deallocating the pointer. It’s covariant w.r.t.Tand affects dropck as inBox<T>.
Since we can use &move T in core, we can write the following FnOnce definition:
trait FnOnce<Args> {
type Output;
fn call_once(self, args: Args) -> Self::Output;
// Similar to `call_once`, but `self` is in an owned reference.
fn call_once_ref(self: &move Self, args: Args) -> Self::Output;
}
Assuming we can obtain &move T from Box<T>, we can now implement Box<FnOnce>: FnOnce.
There may also be a performance advantage: in some cases, one can use &move FnOnce instead of Box<FnOnce>.
Non-builtin owned reference
&move T type seems ideal so far. Nevertheless, attempts to introduce &move T failed several times: rust-lang/rust#965, rust-lang/rust#1617, and rust-lang/rust#1646. (See rust-lang/rfcs#998 for summary) I think this is because it requires too much effort to change the language, the compiler, the standard library, and other libraries for the expected outcome.
Therefore, in this RFC, I propose the non-builtin owned reference type OwnedRef<'a, T> instead of builtin&'a move T. In this way, we can significantly reduce the required effort.
trait FnOnce<Args> {
type Output;
fn call_once(self, args: Args) -> Self::Output;
// Similar to `call_once`, but `self` is in an owned reference.
fn call_once_ref(self: OwnedRef<Self>, args: Args) -> Self::Output;
}
Guide-level explanation
Introduce a new type OwnedRef<'a, T: ?Sized + 'a> to core. OwnedRef is similar to Box<T: ?Sized>, but it is only responsible for freeing the content, not the pointer itself.
Then we change the definition of FnOnce to accept OwnedRef<Self> argument. The result is we can now call Box<FnOnce>.
Once Box<FnOnce> is proved to be working correctly, we can deprecate the use of Box<FnBox>, which is still unstable.
Reference-level explanation
OwnedRef type
OwnedRef is defined as follows:
pub struct OwnedRef<'a, T: ?Sized + 'a> {
inner: Unique<T>,
_marker: PhantomData<&'a ()>,
}
impl<'a, #[may_dangle] T: ?Sized + 'a> Drop for OwnedRef<'a, T> {
fn drop(&mut self) {
unsafe {
mem::drop_in_place(self.inner.as_mut());
}
}
}
Here Unique<T> is a compiler-internal wrapper for NonNull<T>, which is defined as follows:
pub struct Unique<T: ?Sized> {
pointer: NonZero<*const T>,
_marker: PhantomData<T>,
}
unsafe impl<T: Send + ?Sized> Send for Unique<T> {}
unsafe impl<T: Sync + ?Sized> Sync for Unique<T> {}
impl<T: UnwindSafe + ?Sized> UnwindSafe for Unique<T> {}
The semantics of OwnedRef<'a, T> is similar to &'a mut T. However, it is expected that the content (referent) is freed after 'a lifetime.
APIs of OwnedRef at least include the following:
- Extract
TfromOwnedRef<T>whereT: Sized. - Borrow
OwnedRef<T>fromTwhereT: Sized. This API uses callback pattern to ensure lifetime. - Borrow
OwnedRef<T>fromOwnedRef<Box<T>>. This API uses callback pattern to free the pointer after the end of the lifetime. -
Deref,DerefMut, andCoerceUnsizedimpls.
Introduce a new lang item#[lang = "owned_ref"] to allow self: OwnedRef<Self> and ensure its object-safety. The former part is already covered by #![feature(arbitrary_self_types)].
BorrowOwned trait
Add a new trait called BorrowOwned:
pub trait BorrowOwned<Borrowed: ?Sized>: BorrowMut<Borrowed> {
fn borrow_owned<F, R>(self, f: F) -> R
where
F: FnOnce(OwnedRef<Self>) -> R,
Self: Sized,
{ .. }
fn borrow_owned_ref<F, R>(self: OwnedRef<Self>, f: F) -> R
where F: FnOnce(OwnedRef<Self>) -> R;
}
impl<T: ?Sized> BorrowOwned<T> for T { .. }
impl<T: ?Sized> BorrowOwned<T> for Box<T> { .. }
impl<T: ?Sized> BorrowOwned<'a, T: ?Sized + 'a> for OwnedRef<'a, T> { .. }
Unlike BorrowMut, BorrowOwned uses the callback pattern, because BorrowOwned<T> for Box<T> have to free the pointer after the expiration of the owned reference.
Modification to FnOnce trait
Add a new method call_once_ref to FnOnce:
pub trait FnOnce<Args> {
type Output;
extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
extern "rust-call" fn call_once_ref(self: OwnedRef<Self>, args: Args) -> Self::Output;
}
To ensure maximal compatibility (especially of the compiler itself), provide a default implementation of call_once_ref for Self: Sized. It can be done once default impl from RFC 1210 is implemented.
Fix the compiler so that FnOnce can have methods other than call_once. We may also implement call_once_ref for closures.
Drawbacks
Complex borrowing procedure
Borrowing OwnedRef<T> is much more complex than borrowing &mut T and &T. Instead of just writing &mut x or &x, one has to write the following boilerplate:
x.borrow_owned(|owned_ref_to_x| {
// Use owned_Ref_to_x here
})
Built-in owned reference &move T would provide smarter borrowing procedure.
Inflexible owned borrowing
Unlike &mut x and &x borrowing, the compiler doesn’t treat OwnedRef borrowing specially.
For example, we can’t borrow the last field of a dynamically-sized struct/tuple by value.
fn snd(x: &(i32, [u8])) -> &[u8] {
&x.1
}
fn snd_owned(x: OwnedRef<(i32, [u8])>) -> OwnedRef<[u8]> {
// There's no way to borrow by-value the last field!
panic!()
}
Need to change the signature of traits
Unlike by-value DST proposals, owned reference proposals (both built-in and non-builtin) require modification of trait signatures:
pub trait FnOnce<Args> {
type Output;
extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
// Need to add this extra item
extern "rust-call" fn call_once_ref(self: OwnedRef<Self>, args: Args) -> Self::Output;
}
Rationale and alternatives
By-value DST, &move T, and OwnedRef<T> are more elegant and uniform solution than FnBox. Among these, OwnedRef<T> introduces much less complexity to the language, as it is mostly library-level modification. Although OwnedRef<T> would be a bit more inconvenient than by-value DST and &move T, it is not a big problem because most uses of OwnedRef<T> would be in a glue code in a library.
Allow unsized self to be passed by-value
By-value DST (as seen in rust-lang/rfcs#977 and rust-lang/rfcs#90) is a direct approach to Box<FnOnce>, which doesn’t require modification of trait signatures. However, we’ll have to carefully modify the treatment of Sized constraints in both typeck and trans.
Introduce built-in &move T type
The built-in owned reference (as seen in rust-lang/rust#965, rust-lang/rust#1617, and rust-lang/rust#1646) is similar to this RFC, but the type &move T is built in the compiler. As a consequence, we can expect more flexible and easy borrowing. However, it still requires careful modification of typeck and borrowck.
Keep the status quo, stabilizing FnBox
FnBox still has problems. Most notable ones are:
- We can’t use late-bound lifetimes in
FnBoxe.g.Box<FnBox(&str) -> &[u8]>. - There is a coherence problem where the existence of
FnBoxprevents some usefulFn/FnMutimpls.
Additionally, it’s not intuitive to have the two very similar traits: FnOnce and FnBox.
Keep the status quo, without stabilizing FnBox
If we keep the status quo without even stabilizing FnBox, there is no stable and general way to wrap FnOnce in a trait object. I think this is not the right direction to go, since there are certainly demands to box FnOnce in a trait object.
Unresolved questions
There are no unresolved questions so far.