The problem
Methods that take type parameters are not object safe. Using the upcoming Allocator API requires adding allocator type parameters everywhere. Using custom allocators with object-safe traits is therefore a hassle. This issue has been brought up here, among other places.
To summarize, we'd like the following trait to be object-safe:
trait Foo {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A>;
}
Because of the A type parameter, this isn't possible today. And because A appears in the return type, it still wouldn't be possible under proposals for limited APIT in trait objects.
MVP workaround
One possible workaround is to use &dyn Allocator instead of a type parameter, like the below:
trait Foo {
fn frobnify<'a>(&self, a: &'a dyn Allocator) -> Vec<i32, &'a dyn Allocator>;
}
Unfortunately, this will infect our entire API with &'a dyn Allocator and its associated lifetime. But we can do better. With into_raw_parts and from_raw_parts_in, we can switch between static and dynamic dispatch for Allocator. Combining this with a separate helper trait, we can greatly improve the situation:
#![feature(allocator_api)]
#![feature(vec_into_raw_parts)]
use std::alloc::Allocator;
// This is not object-safe...
trait Foo {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A>;
}
// But this is!
trait DynFoo {
fn frobnify_dyn<'a>(&self, a: &'a dyn Allocator) -> Vec<i32, &'a dyn Allocator>;
}
impl<T: Foo> DynFoo for T {
fn frobnify_dyn<'a>(&self, a: &'a dyn Allocator) -> Vec<i32, &'a dyn Allocator> {
<Self as Foo>::frobnify(self, a)
}
}
impl<'a> Foo for dyn DynFoo + 'a {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A> {
let dyn_a: &dyn Allocator = &a;
let v: Vec<i32, &dyn Allocator> = self.frobnify_dyn(dyn_a);
let (ptr, len, cap) = v.into_raw_parts();
// Safety: the vec was originally allocated with `dyn_a`,
// which was derived from `a`
unsafe { Vec::<i32, A>::from_raw_parts_in(ptr, len, cap, a) }
}
}
// Usage example
use std::alloc::Global;
struct Bar;
impl Foo for Bar {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A> {
let mut v = Vec::new_in(a);
v.push(3);
v
}
}
fn main() {
let dyn_foo: &dyn DynFoo = &Bar;
let v: Vec<i32> = dyn_foo.frobnify(Global);
dbg!(v);
}
This gets us all the functionality we want, but the ergonomics are not ideal.
Getting rid of unsafe
A closure-based API could help encapsulate the unsafe around from_raw_parts_in. For example:
#![feature(allocator_api)]
#![feature(vec_into_raw_parts)]
pub mod dyn_alloc_stuff {
use std::{alloc::Allocator, marker::PhantomData};
#[repr(transparent)]
pub struct BrandedAlloc<'brand> {
// For invariance
brand: PhantomData<fn(&'brand ()) -> &'brand ()>,
alloc: &'brand dyn Allocator,
}
// SAFETY: We just forward to the underlying dyn
unsafe impl<'brand> Allocator for BrandedAlloc<'brand> {
#[inline]
fn allocate(
&self,
layout: std::alloc::Layout,
) -> Result<std::ptr::NonNull<[u8]>, std::alloc::AllocError> {
self.alloc.allocate(layout)
}
#[inline]
unsafe fn deallocate(&self, ptr: std::ptr::NonNull<u8>, layout: std::alloc::Layout) {
self.alloc.deallocate(ptr, layout)
}
#[inline]
fn allocate_zeroed(
&self,
layout: std::alloc::Layout,
) -> Result<std::ptr::NonNull<[u8]>, std::alloc::AllocError> {
self.alloc.allocate_zeroed(layout)
}
#[inline]
unsafe fn grow(
&self,
ptr: std::ptr::NonNull<u8>,
old_layout: std::alloc::Layout,
new_layout: std::alloc::Layout,
) -> Result<std::ptr::NonNull<[u8]>, std::alloc::AllocError> {
self.alloc.grow(ptr, old_layout, new_layout)
}
#[inline]
unsafe fn grow_zeroed(
&self,
ptr: std::ptr::NonNull<u8>,
old_layout: std::alloc::Layout,
new_layout: std::alloc::Layout,
) -> Result<std::ptr::NonNull<[u8]>, std::alloc::AllocError> {
self.alloc.grow_zeroed(ptr, old_layout, new_layout)
}
#[inline]
unsafe fn shrink(
&self,
ptr: std::ptr::NonNull<u8>,
old_layout: std::alloc::Layout,
new_layout: std::alloc::Layout,
) -> Result<std::ptr::NonNull<[u8]>, std::alloc::AllocError> {
self.alloc.shrink(ptr, old_layout, new_layout)
}
}
pub fn make_vec_with_dyn<T, A, F>(a: A, f: F) -> Vec<T, A>
where
A: Allocator,
F: for<'brand> FnOnce(BrandedAlloc<'brand>) -> Vec<T, BrandedAlloc<'brand>>,
{
let brand_a: BrandedAlloc<'_> = BrandedAlloc {
brand: PhantomData,
alloc: &a,
};
let v = f(brand_a);
let (ptr, len, cap) = v.into_raw_parts();
// Safety: the vec was originally allocated with `brand_a`,
// which was derived from `a`
unsafe { Vec::<T, A>::from_raw_parts_in(ptr, len, cap, a) }
}
}
use std::alloc::Allocator;
use dyn_alloc_stuff::*;
trait Foo {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A>;
}
trait DynFoo {
fn frobnify_brand<'brand>(&self, a: BrandedAlloc<'brand>) -> Vec<i32, BrandedAlloc<'brand>>;
}
impl<T: Foo> DynFoo for T {
fn frobnify_brand<'brand>(&self, a: BrandedAlloc<'brand>) -> Vec<i32, BrandedAlloc<'brand>> {
<Self as Foo>::frobnify(self, a)
}
}
impl<'a> Foo for dyn DynFoo + 'a {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A> {
make_vec_with_dyn(a, |dyn_a| self.frobnify_brand(dyn_a))
}
}
But one could end up needing many variants of make_vec_with_dyn, for all the different collection types (may be abstractable with GATs), and whether you are creating or mutating them. And maybe you are working with multiple collections at the same time... Perhaps you have a better idea?
Getting rid of DynFoo
Ideally dyn DynFoo would just be dyn Foo. But this would require language changes. Perhaps something like the below?
trait Foo {
// Add an entry to the `dyn Foo` vtable for a particular monomorphization
// of this function. It becomes an inherent method of `dyn Foo`.
#[include_in_vtable(frobnify_dyn::<&dyn Allocator>)]
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A>;
}
// Provide the missing methods of the trait object.
impl<'a> Foo for dyn Foo + 'a {
fn frobnify<A: Allocator>(&self, a: A) -> Vec<i32, A> {
let dyn_a: &dyn Allocator = &a;
let v: Vec<i32, &dyn Allocator> = self.frobnify_dyn(dyn_a);
let (ptr, len, cap) = v.into_raw_parts();
unsafe { Vec::<i32, A>::from_raw_parts_in(ptr, len, cap, a) }
}
}
Miscellaneous questions
- How would a
StorageAPI affect all this? - Is
dynx/dyn*relevant here?