()`.
~~(note: currently I have a bug where I don't check the alignments, but that will soon be fixed)~~ now fixed
---
### Author:  [@TimDiekmann](https://internals.rust-lang.org/u/TimDiekmann)
#### Post date: [December 4, 2020, 2:38pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/49 "2020-12-04T14:38:16Z")
I'll look into your implementation tomorrow. I think I have a solution to this, too but let's see...
Hopefully I'll find enough spare time to push this forward, it's by far the most promising proposal for collections I have seen!
* * *
> [@carbotaniuman](#):
>
> There's already the Allocator WG, so I don't think an RFC is really needed here - just do some exploration for the WG first without worrying about all the bureaucracy.
I [filed an issue](https://github.com/rust-lang/wg-allocators/issues/79) in the WG repository.
* * *
# Update:
I updated my crate to [v0.1.0](https://docs.rs/storages/0.1.0). It's may worth a look at [`RawBox`](https://docs.rs/storages/0.1.0/storages/boxed/struct.RawBox.html) and [`Box`](https://docs.rs/storages/0.1.0/storages/boxed/struct.Box.html). Main features:
- typed buffers with only two traits ([`Buffer`](https://docs.rs/storages/0.1.0/storages/buffer/trait.Buffer.html) and [`UnmanagedBuffer`](https://docs.rs/storages/0.1.0/storages/buffer/trait.UnmanagedBuffer.html))
- [`AllocatedBuffer`](https://docs.rs/storages/0.1.0/storages/buffer/struct.AllocatedBuffer.html) provided
- native arrays are supported
- Two APIs: raw API with call-side-dependency-injection and non-raw wrapper API, which behaves like the current api
- Coercion is working
- Unsized types are working
---
### Author:  [@adamnemecek](https://internals.rust-lang.org/u/adamnemecek)
#### Post date: [December 8, 2020, 3:54am UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/50 "2020-12-08T03:54:22Z")
@TimDiekmann looking at the Storage trait, maybe we could consider adding AsPtr and AsMutPtr traits to the std. It feels cleaner than the Storage trait.
---
### Author:  [@TimDiekmann](https://internals.rust-lang.org/u/TimDiekmann)
#### Post date: [December 8, 2020, 10:41am UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/51 "2020-12-08T10:41:32Z")
I am not sure if it is worth it. Considering that pointer dereferencing is generally `unsafe` and `Storage` (or `Buffer` as it is called in my crate) is just a backend that the usual user will rarely interact with, I think a `Buffer` trait is more flexible. Of course, an `AsPtr`-trait could make sense in other places too, but I don't know how well it is backwards compatible.
I would stick with the `Buffer`-trait for now.
---
### Author:  [@carbotaniuman](https://internals.rust-lang.org/u/carbotaniuman)
#### Post date: [December 8, 2020, 6:47pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/52 "2020-12-08T18:47:53Z")
How does this decide whether to use internal storage or the heap storage?
---
### Author:  [@Ericson2314](https://internals.rust-lang.org/u/Ericson2314)
#### Post date: [December 9, 2020, 2:10am UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/53 "2020-12-09T02:10:27Z")
I do fee likel @matklad's idea is a reason to switch back to `&mut self` or even `self` for allocators. With the allocators passed in at the last moment rather than stored, there is no longer the issue of storing allocators for multiple collections in a single threaded context. The dreaded `AllocOnce` `AllocMut` `Alloc` hierarchy can also come up based on whether it's a flat or non-flat data structure.
The main issue is it is also nice to pass in at deallocation time, but how to we prevent dropping without the allocator being passed in? This was my #1 reason for wanting linear types, years ago when I clamored about these things.
---
### Author:  [@matthieum](https://internals.rust-lang.org/u/matthieum)
#### Post date: [December 9, 2020, 5:02pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/54 "2020-12-09T17:02:51Z")
This is massively invasive, requiring to change every single API, and requiring users to follow a very strict discipline since the same allocator _must_ be passed to every single call -- turning every single function into an `unsafe` function OR mandating that allocators be able to recognize their own allocations. Honestly, it seems completely impractical.
Note that even in Zig, where there is no global allocator, collections tend to capture the allocator in their constructor and use it throughout, including for destruction.
If you want to start exploring the possibility, I would kindly ask that you open a separate topic as your proposal is at the moonshot stage (not even a mini-draft/prototype, no experience that I can see) which means it'll require significant brainstorming to go anywhere.
---
### Author:  [@matklad](https://internals.rust-lang.org/u/matklad)
#### Post date: [December 9, 2020, 6:44pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/55 "2020-12-09T18:44:23Z")
Another interesting point here is that in Zig it's easier to handle "bring your own allocator"-style collections, because it uses value-based defer, rather than type-based destructors. So you, eg, can close over a single allocator local var to drop to local vars holding vectors.
---
### Author:  [@Wodann](https://internals.rust-lang.org/u/Wodann)
#### Post date: [December 24, 2020, 10:29am UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/56 "2020-12-24T10:29:56Z")
@TimDiekmann, have you been able to make headway with your reference implementation? It seems like an interesting concept to explore.
Thanks to @matthieum for raising this before we finalised the allocator API. It's always valuable to explore new avenues, at any point in the development process 😄
---
### Author:  [@TimDiekmann](https://internals.rust-lang.org/u/TimDiekmann)
#### Post date: [December 27, 2020, 12:53am UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/57 "2020-12-27T00:53:22Z")
I wasn't able to spend much more time recently so the repository is at the latest state. I'll investigate this further as soon as I find some spare time. 🙂
---
### Author:  [@matthieum](https://internals.rust-lang.org/u/matthieum)
#### Post date: [December 28, 2020, 4:34pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/58 "2020-12-28T16:34:03Z")
I've been battling with trait representations.
It's easy enough to get the size and pointer of a slice, store the size and content separately, then reassemble them on the fly, however I could not find a (good) API for traits -- using `transmute_copy` on the unstable `TraitObject` representation is rather error-prone -- and more importantly I could not find a generic way to take a `?Sized` object, break it down into meta-data + data, and reassemble it.
The closest I've found is the `thin` crate, and by default it only handles a couple of traits (by implementing a custom trait for them).
I'm thinking we may be lacking a more fundamental API on which `RawBox` can be built.
---
### Author:  [@RustyYato](https://internals.rust-lang.org/u/RustyYato)
#### Post date: [December 28, 2020, 4:39pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/59 "2020-12-28T16:39:22Z")
> [@matthieum](#):
>
> and more importantly I could not find a generic way to take a `?Sized` object, break it down into meta-data + data, and reassemble it.
There isn't one! That's what all these open RFCs are trying to achieve
>
>
> Those two new language traits are at the intersection of #1861, #2580, #2594, an…d this RFC's raison d'être is solely to try to get some progress for those 3 RFCs all at once by specifying their common core part.
>
> \[Rendered\](https://github.com/nox/rust-rfcs/blob/patch-1/text/0000-pointee.md)
>
>
> This has been a long time coming - similar to #2580, but more general.
>
> ~~On t…he unresolved question about \`?Sized -\> DynamicallySized\`, and the need for \`?DynamicallySized\`. I think this is actually a reasonable change which makes \`extern type\`s act much nicer around generic functions that already exist, and fixes all the Rust code that has been written already. No longer is, for example, \`Box\\` allowed.~~ I've chosen to make this change.
>
> Note that \`DynamicallySized\` is a new language trait, on the level of \`Sized\` and \`Copy\`.
>
> \[Rendered\](https://github.com/ubsan/rust-rfcs/blob/custom-dst/text/0000-custom-dst.md)
>
>
> Add generic APIs that allow manipulating the metadata of fat pointers:
>
> \* Nami…ng the metadata’s type (as an associated type)
> \* Extracting metadata from a pointer
> \* Reconstructing a pointer from a data pointer and metadata
> \* Representing vtables, the metadata for trait objects, as a type with some limited API
>
> This RFC does \*not\* propose a mechanism for defining custom dynamically-sized types, but tries to stay compatible with future proposals that do.
>
> \[Rendered\](https://github.com/rust-lang/rfcs/blob/master/text/2580-ptr-meta.md)
Even some closed ones
>
>
> \[Rendered\](https://github.com/kennytm/rfcs/blob/dyn-sized/text/0000-dyn-sized.md…)
>
> cc #2255
> cc @mikeyhew, @mystor (#1993), @withoutboats
>
>
> I believe this fixes #813, and is a nicer, and far more powerful, solution than …#709.
---
### Author:  [@matthieum](https://internals.rust-lang.org/u/matthieum)
#### Post date: [December 29, 2020, 12:57pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/60 "2020-12-29T12:57:33Z")
I left a comment on [RFC 2984](https://github.com/rust-lang/rfcs/pull/2984#issuecomment-752065665) which I think is the most minimal.
A straightforward extension to the RFC is all it takes to be able to break down a pointer into its raw parts (meta-data and pointer to data) and stitch them back together afterwards.
---
### Author:  [@matthieum](https://internals.rust-lang.org/u/matthieum)
#### Post date: [January 3, 2021, 2:04pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/61 "2021-01-03T14:04:05Z")
I'd like to point to you [Coca's author experiments](https://gist.github.com/teryror/7b9a23fd0cd8dcfbcb6ebd34ee2639f8).
Of particular interest is the use of the `set_ptr_value` feature to reset the data-pointer of a fat-pointer. By using it, @teryror manages to handle `Sized`, slices, and traits in a generic manner (preserving their meta-data).
It's not ideal as the original pointer is dead-weight, adding up an extra 8 bytes, however it's working right now which is nice to further explore the API design.
---
### Author:  [@Ericson2314](https://internals.rust-lang.org/u/Ericson2314)
#### Post date: [January 7, 2021, 11:32pm UTC](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460/62 "2021-01-07T23:32:33Z")
> [@matklad](#):
>
> values-based defer
Oh....shit. I spent ages haranguing people about how we would need linear types for this issue. But linear types that simply require all bindings be `defer`-ed rather than only working with out unwinding would probably get much less push-back, and are _exactly_ what this needs.
(To be precise, it's the unwinding that requires the defer, not the linear-typed values themselves, but people were and presumably are uncomfortable with no-unwinding changing the language that deeply, so I won't go there.)
I most certainly do not have the bandwidth to write this RFC, but someone else probably should. It will make custom allocator stuff 100x better.
[Previous page](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460.md?page=2)
[Next page](https://internals.rust-lang.org/t/is-custom-allocators-the-right-abstraction/13460.md?page=4)