Possible to un-special-case Box<T>?

[H2CO3]

I wasn’t able to find a related proposal, RFC, or issue on GitHub, so I’d like to ask here: would it be possible to transform Box<T> into a true library-implemented type?

Currently, Box<T> is a special-cased, quite ugly language item. As I understand, this has historical reasons, because originally, heap-allocated uniques were a built-in type, but they no longer are. This leads to issues you probably know very well. A non-exhaustive list of symptoms:

• The Box constructor is magic. Box::new(value) is literally implemented as box value. (This already feels backwards; I’d expect box syntax to just be sugar for Box::new().)
• The destructor is also magic: Box::drop() is a no-op with a comment saying that the real work is performed by the compiler.
• The type needs (?), but certainly has, special handling in the compiler, increasing the complexity of internals for I think no good reason.
• Relatedly, impl Deref is confusing, it looks as if it was infinitely recursive whereas it’s not.
• Relatedly, *some_box moves out of the box and deallocates. This is inconsistent with the behavior of other Deref and smart pointer types in the ecosystem.
• It’s mildly inconvenient to implement Box<T> yourself (if someone were so inclined), because it needs the exchange_malloc and exchange_free lang items.
• And generally, it just feels wrong that such a simple type isn’t implemented in pure Rust, like C++'s std::unique_ptr is implemented in C++.

Given all these drawbacks, wouldn’t it be better to make Box a regular library type? Is it possible? Today’s Rust seems to have all the necessary features for doing something like:

struct Box<T: ?Sized>(*mut T);

impl<T> Box<T> {
    fn new(value: T) -> Self {
        // IRL we wouldn't libc in std, of course
        let ptr = libc::malloc(mem::size_of::<T>()) as *mut T;
        let garbage = mem::replace(unsafe { &mut *ptr }, value);
        mem::forget(garbage);
        Box(ptr)
    }
}

impl<T> Drop for Box<T> {
    fn drop(&mut self) {
        let value = unsafe { mem::replace(&mut *self.0, mem::uninitialized() };
        drop(value);
        libc::free(self.0 as *mut c_void);
    }
}

impl<T> Deref for Box<T> {
    type Target = T;

    fn deref(&self) -> &T {
        unsafe { &*self.0 }
    }
}

etc.

I understand that there may be scenarios when special knowledge about a box is required, but couldn’t only those requirements be moved into a trait that itself is a lang item, leaving the rest of poor Box impl alone?

[arielb1]

You can move things out of a Box and the compiler-generated drop impl will only drop the box itself afterwards. You can’t do this with a current smart pointer. There’s a DerefPure proposal to make it work, but it’s still an RFC.

fn foo(x: Box<String>) -> String {
    let data = *x; // this moves the `String` away

    // and here, at end of scope, the `Box` is freed but not the string
    data
}

The box syntax uses placement new to avoid some redundant stack copies. We could change it to use the new library placement new API instead of being known to the compiler and I believe there’s a plan for doing so. In that case Box::new would still call it.

[BenoitJGirard]

I think there is also some magic involved in coercing a Box<T> to a “boxed trait object” Box<TraitOfT>. (Update: I did not know about CoerceUnsized.)

One day Rust will have to tackle local/arena memory allocators, which are needed in some problem domains for performance; these will require box types that don’t use the global heap, are not Send, etc.

Here’s a 2-part presentation from CppCon on these types of allocators:

[josh]

crates.io includes several different implementations of arena allocators with various properties. Might one of those do what you’re looking for?

[BenoitJGirard]

As discussed above, no library smart pointer type can do what Box<T> does, because Box<T> is “magic”.

So whatever libraries currently exist for arena allocators, they do not and cannot provide a full, Rust-idiomatic heap experience, with a smart pointer type as complete as Box<T>.

[hyeonu]

Problem is, Rust doesn’t have DerefMove yet. There’s an issue for it, but this issue is tied with other traits like IndexMove which need more discussions how should they work.

However, I don’t think we need more discussions for DerefMove itself. Maybe we should make another independent RFC for it?

[H2CO3]

There are two traits, Unsize and CoerceUnsized, which in my understanding are suited exactly for converting any generic SmartPointer<T> to SmartPointer<Trait>. IMO it would be sufficient to have these traits as magic lang items, instead of some concrete type.