# Idea: introduce core::ptr::read\_owned **URL:** \ **Category:** language design\ **Created:** [June 21, 2023, 4:14pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038 "2023-06-21T16:14:41Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 21, 2023, 4:14pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/1 "2023-06-21T16:14:41Z")
I have a relatively large type `Foo` which has consuming method `foo`. In my code I work with `*mut Foo` which semantically "owns" the underlying `Foo` value (i.e. we can safely call [`drop_in_place`](https://doc.rust-lang.org/beta/core/ptr/fn.drop_in_place.html) on it). I want to call `foo` for value behind this pointer. I can write something like this: ```rust let p: *mut Foo = ...; let val = core::ptr::read(p); val.foo(); ``` This code works, but it has a big problem: compiler can not modify the data behind `p`, thus it has to copy `Foo` onto stack first and then call `foo` on the copied data. Obviously, it's not efficient for large types. I wonder if it's feasible to add something like: ```rust /// Reads the value from `src` by moving it. /// /// The memory behind `src` must not be read or written before the /// return value is dropped. If `T` is not `Copy`, reading the memory after /// the return value is dropped can cause undefined behavior. pub unsafe fn read_owned(src: *mut T) -> T { ... } ``` It would be equivalent to pointer dereferencing, but without the `Copy` bound. ---
**Author:** ![197g](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/197g/32/7276_2.png) [@197g](https://internals.rust-lang.org/u/197g)\ **Post date:** [June 21, 2023, 5:33pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/2 "2023-06-21T17:33:23Z")
What might also work is a form of builtin owning pointer type here. Maybe that best parallel we have for this is `Box`. By builtin magic it's somehow possible for op-call to consume that by-value despite only having a pointer, as an unsized type. In general, a pointer owning the value but not its allocation is something still missing from the core language. Sometimes that's called stack-box or similar in crates. Maybe it'd would be possible to have as a builtin motivated by providing the ability to call methods that consume its value, somehow without those methods needing to be aware, or naming it as the receiver type? (or via `DerefTake` but that seems quite a huge feature to introduce at the same time). ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 21, 2023, 7:50pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/3 "2023-06-21T19:50:08Z")
Yeah, something like `*own Foo` would work as well. It also could be a good hint for C APIs written in Rust. Maybe it could be implemented as `*mut Foo`, but with allowed dereferncing of non-`Copy` types? ---
**Author:** ![scottmcm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/scottmcm/32/2355_2.png) [@scottmcm](https://internals.rust-lang.org/u/scottmcm)\ **Post date:** [June 21, 2023, 10:20pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/4 "2023-06-21T22:20:09Z")
> [@newpavlov](#): > > This code works, but it has a big problem: compiler can not modify the data behind `p`, thus it has to copy `Foo` onto stack first and then call `foo` on the copied data. Obviously, it's not efficient for large types. The call to `ptr::read` compiles to `val = copy *p;` in MIR [now](https://github.com/rust-lang/rust/pull/109035). I wonder if we could largely address this with a `read_move` that compiles to `val = move *p` in MIR, so that MIR optimizations could then just reuse the `*p` place instead of actually making the copy to the temporary. But that might get entangled in the "what can you do after moving out of something" conversation... ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 21, 2023, 11:06pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/5 "2023-06-21T23:06:01Z")
> [@scottmcm](#): > > But that might get entangled in the "what can you do after moving out of something" conversation... I think semantics should be similar to `drop_in_place`, no? After you moved data out of pointer, the memory behind this pointer effectively becomes uninitialized, i.e. you can not read it before writing a new value into it. ---
**Author:** ![WorldSEnder](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/worldsender/32/9673_2.png) [@WorldSEnder](https://internals.rust-lang.org/u/WorldSEnder)\ **Post date:** [June 22, 2023, 12:00am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/6 "2023-06-22T00:00:45Z")
> [@newpavlov](#): > > compiler can not modify the data behind `p`, thus it has to copy `Foo` onto stack first and then call `foo` on the copied data. That is because it has to assume that you wanted to leave a valid bit pattern for `Foo` in whatever was pointed to by that pointer. To allow you to move destructively out of that place, it should be possible to let the caller pass in a `*mut MaybeUninit` instead. I think a better approach is outlined in [this gist](https://gist.github.com/WorldSEnder/e6d4bebe8cc5a03c84c791ec81712c7f#file-surely_init-rs). We have a `SurelyInit` struct which encapsulates the guarantee that a specific `MaybeUninit` is initialized. Crucially, it has a `take()` method which moves out that MaybeUninit and leaves behind whatever garbage the compiler wants to (but marks the memory uninitialized). You would then write your method consuming the data, passing a ` SurelyInit<'_, LargeStruct>` instead of `*mut LargeStruct`: ```rust fn consume(place: SurelyInit<'_, LargeStruct>) -> u8 { let val = place.take(); // This might look like it moves the struct // in reality, the copy is avoided under mild optimizations } ``` [Godbolt example to show assemly](https://gcc.godbolt.org/z/cf8EeM9bE). ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 22, 2023, 12:31am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/7 "2023-06-22T00:31:52Z")
> [@WorldSEnder](#): > > That is because it has to assume that you wanted to leave a valid bit pattern for `Foo` in whatever was pointed to by that pointer. To allow you to move destructively out of that place, it should be possible to let the caller pass in a `*mut MaybeUninit` instead. Raw pointers to `MaybeUninit` are virtually useless. Raw pointers do not provide any guarantees about validity. They can easily point to uninitialized or unaligned memory, or even be null pointers. Your `mem::replace` trick also does not work properly as can be demonstrated by this snippet: [Compiler Explorer](https://rust.godbolt.org/z/z9KjYY9Gr) (the same [happens](https://gcc.godbolt.org/z/fa7eKG7dd) with your snippet as well). As noted by @scottmcm the problem is simply that the language currently does not expose an operation with the necessary semantics. ---
**Author:** ![WorldSEnder](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/worldsender/32/9673_2.png) [@WorldSEnder](https://internals.rust-lang.org/u/WorldSEnder)\ **Post date:** [June 22, 2023, 12:56am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/8 "2023-06-22T00:56:50Z")
> [@newpavlov](#): > > As noted by @scottmcm the problem is simply that the language currently does not expose an operation with the necessary semantics. Seems I was under the mistaken assumption that `mem::replace` was sufficient here. But I don't understand why my original godbolt example works as intended (generates equal assembly), but taking a `&mut` to the data taken out of the `MaybeUninit` and passing that ref/pointer to some other function seems to force a mem-copy, see [commenting/uncommenting this marked line](https://gcc.godbolt.org/z/KKxsbqcoo) ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 22, 2023, 1:04am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/9 "2023-06-22T01:04:01Z")
It's because your consumption code does not perform any mutation, so compiler is able to use the "replaced" memory directly. Even a simple side-effect free mutation can trigger insertion of `memcpy`: [Compiler Explorer](https://gcc.godbolt.org/z/e7frjqM6b) ---
**Author:** ![scottmcm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/scottmcm/32/2355_2.png) [@scottmcm](https://internals.rust-lang.org/u/scottmcm)\ **Post date:** [June 22, 2023, 5:15am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/10 "2023-06-22T05:15:51Z")
> [@newpavlov](#): > > After you moved data out of pointer, the memory behind this pointer effectively becomes uninitialized, i.e. you can not read it before writing a new value into it. This is not quite what happens today, though. It _can't_ be what `drop_in_place` does, because [`ManuallyDrop::drop`](https://doc.rust-lang.org/nightly/std/mem/struct.ManuallyDrop.html#method.drop) exists, and `ManuallyDrop` preserves niches, which means that it must preserve validity invariants, which means that `drop_in_place` overwriting the memory with `undef` can't happen because that would be UB. ---
**Author:** ![SkiFire13](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/skifire13/32/7579_2.png) [@SkiFire13](https://internals.rust-lang.org/u/SkiFire13)\ **Post date:** [June 22, 2023, 7:12am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/11 "2023-06-22T07:12:58Z")
> [@newpavlov](#): > > I think semantics should be similar to `drop_in_place`, no? After you moved data out of pointer, the memory behind this pointer effectively becomes uninitialized, i.e. you can not read it before writing a new value into it. That doesn't prevent other code from writing to it though, overwriting the value that was supposed to be moved out. To prevent this you would also need the same semantics of `&mut T`, which guarantees it will be the only pointer from which writes are allowed. ---
**Author:** ![197g](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/197g/32/7276_2.png) [@197g](https://internals.rust-lang.org/u/197g)\ **Post date:** [June 22, 2023, 10:29am UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/12 "2023-06-22T10:29:28Z")
Right, that's what 'stack box' usually does. For instance, some code I wrote a few months back: [LeakBox in static\_alloc::leaked - Rust](https://docs.rs/static-alloc/latest/static_alloc/leaked/struct.LeakBox.html#method.from_mut) . With several unsafe methods for passing a value as a mutable reference such as `from_raw` for a pointer etc. This usually requires promising that there are no other underlying mutable references active after the call (i.e. it is sound to deinitialize the value, similar to `ManuallyDrop::take`). However, for some types it is not actually necessary. For instance, using `MaybeUninit` is always sound and so are `T: Copy` types. (All types that would be sound as union fields, but that's just an unproven observation!). This is because 'dropping' these types does not _actually_ invalidate any other later read from that place. The MU in particular makes it possible to provide a parallel to `Box::assume_init` and its safe alternative, which provides a fully safe path for creating the type if the underlying _place_ was declared with `MU` as its type. That is, I think, very very cute. There's a few other observations about soundness of such a wrapper struct hidden in the documentation with confusing naming.. If added to std the implementation could be switched from `NonNull` to `Unique` for the pointer, too, thus resembling a normal `&mut _` more closely. ---
**Author:** ![jrose](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jrose/32/9591_2.png) [@jrose](https://internals.rust-lang.org/u/jrose)\ **Post date:** [June 22, 2023, 5:17pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/13 "2023-06-22T17:17:20Z")
[std::mem::needs\_drop](https://doc.rust-lang.org/std/mem/fn.needs_drop.html) is probably close to the right condition, including the part where it says “this can conservatively return `true` for everything”. ---
**Author:** ![197g](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/197g/32/7276_2.png) [@197g](https://internals.rust-lang.org/u/197g)\ **Post date:** [June 22, 2023, 6:58pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/14 "2023-06-22T18:58:06Z")
Expanding the condition form `Copy` to `!needs_drop()` seems a little rushed, and I'm not sure the reason. For the `Copy` trait the absence of `Drop` is a compile guarantee, for the function it is probably somewhat right but I would not currently use it for that purpose. In particular, we're implicitly passing a (retained) value of `T` back to the owner of the place after `*own ` is dropped. There should be a justification where it comes from and this is _not_ explicitly what `needs_drop` provides when returning `false`. ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 26, 2023, 6:03pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/15 "2023-06-26T18:03:12Z")
> [@scottmcm](#): > > I wonder if we could largely address this with a `read_move` that compiles to `val = move *p` in MIR, so that MIR optimizations could then just reuse the `*p` place instead of actually making the copy to the temporary. Unfortunately, it does not work: [Add `ptr::read_move` by newpavlov · Pull Request #113066 · rust-lang/rust · GitHub](https://github.com/rust-lang/rust/pull/113066) ---
**Author:** ![197g](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/197g/32/7276_2.png) [@197g](https://internals.rust-lang.org/u/197g)\ **Post date:** [June 26, 2023, 8:58pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/16 "2023-06-26T20:58:13Z")
(Edit: below are somewhat raw thoughts on whether the problem is a deep property of places, the notion how places are fundamentally created, how they are named, and where Rust _requires_ creating a place but maybe it should not. I think I'm still trying to really understand where the problem of the thread comes from in the first place, before finding a solution for it). There's a pretty fundamental property in a lot of the high level semantics: places are disjunct when their declaration points differ. (Which makes the temporary _value_ sequence points in dereference chains somewhat devious to pure 'place pivoting' / place naming parts of expressions). For that reason an assignment like that always creates a conceptually _new_ place and optimizations will definitely want to exploit the disjuncitve property. Trying to do this via any intermediate form that does declare a new place will thus require very carefully revisting each optimization which is probably infeasible. This disjunctive property is even much stricter than `noalias`, since it's a fact not about the current borrow/tag stack but on the allocations underneath. (Even though I don't think such a fully proper attribute is available in llvm currently so maybe it's feasible to solve in HIR/MIR for the moment. That would bar adding some of these attributes though for function attributes). Your syntax request seems to have two major effects to me: - Change the compiler's initialization bool of an existing place, i.e. treat it as partially moved of necesary. This is also what moves the value ownership. - Introduce a new name for that partial place that _will_ alias with existing places. (though that existing one may be a temporary one in the case of `*raw_ptr`). A bit of that is also fairly engrained in surface language. Some syntax only works by creating new places in the semantics. I'd imagine the above would only be ergonomic if all of these instances had some alternative form where one could provide (and consume/fill) such an owning reference to an existing place instead, in order to keep the substitution principle somewhat working. Maybe I'm overestimating the problem, a look at the syntax reference would be helpful. In particular: - function arguments are always new places with respect to the caller scope. - return values are always new places, but there is actually quite a lot of motivation for relaxing this: multiple return values, construct-in-place, multiple return paths could all potentially be enabled? The details would need to be fleshed out though. - other value-returning expressions, in particular struct construction is always into a new place, which makes DSTs 'inconstrictible'. This overlap for this inverse operation to yours is somewhat interesting? Sure, it would be useful if we could decouple the concept of `value` from its underlying place where the language currently mixes it quite often, as in function parameters. Incidentally, providing more explicit ways to manage places distinct from the values they _might_ contain would also relate to in-place initialization as the inverse as well as multiple return values. But maybe I'm just hallucinating some unfulfillable mess of language features working together 😃 ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 26, 2023, 9:27pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/17 "2023-06-26T21:27:31Z")
Honestly, I am not knowledgeable enough about language semantics and compiler internals, so most of your post goes over my head. 🙂 The thing I want is to efficiently bridge unsafe pointer-based code (e.g. in FFI or certain `unsafe` wizardry) with consuming functions and methods implemented in safe Rust. I know that ABI-wise the compiler passes large arguments as pointers even if in Rust code they are passed as values. So if we have an "owning" pointer `*own T`, I would like to use it in place of `T`, then depending on function ABI the compiler would either use the pointer as-is, or will insert implicit read. I know that there is a whole world of language semantics into which we have to fit this feature, but, unfortunately, I can not make meaningful suggestions in this area. ---
**Author:** ![Vorpal](https://avatars.discourse-cdn.com/v4/letter/v/aca169/32.png) [@Vorpal](https://internals.rust-lang.org/u/Vorpal)\ **Post date:** [June 26, 2023, 9:41pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/18 "2023-06-26T21:41:14Z")
I'm not a language expert, just a somewhat novice user, but isn't it possible to work around this whole issue using a wrapper that holds the pointer internally? This wrapper could be consumed, with the proper semantics, calling into underlying unsafe methods on the inner type. This seems similar to a smart pointer to me. If [arbitrary self types](https://doc.rust-lang.org/beta/unstable-book/language-features/arbitrary-self-types.html) was stabilised, then support for generic such smart pointers with consuming behaviour could be implemented directly on the inner type. As of right now I think you need the logic on the outer wrapper type. ---
**Author:** ![197g](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/197g/32/7276_2.png) [@197g](https://internals.rust-lang.org/u/197g)\ **Post date:** [June 26, 2023, 9:46pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/19 "2023-06-26T21:46:15Z")
The largest contention point is only being able to call _existing_ and independently defined by-value functions (including, via an fn-value where everything must work with the existing ABI for such a function). Maybe we just compare the current LLVM-IR for that. ```llvm ; struct Large { val: [u8; 512} } ; fn test(_: Large) define void @_ZN10playground4test17h78c2f8a40f6d0b1dE(ptr noalias nocapture noundef dereferenceable(512) %val) unnamed_addr #1 personality ptr @rust_eh_personality ``` ```llvm ; fn test2(_: &mut Large) or ABI equivalent type define void @_ZN10playground5test217h1bf6053f48c8c597E(ptr noalias noundef align 1 dereferenceable(512) %val) unnamed_addr #1 { ``` It definitely looks like the 'owning pointer' could be passed to the by-value function directly with suitable language primitives to enable this confounding of nominally different argument types and the proper translation (read: move) for types which are not passed by-pointer in the ABI. At least for now the signature has no attributes with caller-requirements beyond those for a normal `&mut _`. Only `nocapture` which is a requirement on the callee. ---
**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\ **Post date:** [June 26, 2023, 9:49pm UTC](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038/20 "2023-06-26T21:49:55Z")
> [@Vorpal](#): > > This seems similar to a smart pointer to me. If [arbitrary self types](https://doc.rust-lang.org/beta/unstable-book/language-features/arbitrary-self-types.html) was stabilised, then support for generic such smart pointers with consuming behaviour could be implemented directly on the inner type. As of right now I think you need the logic on the outer wrapper type. Consuming functions/methods are external to my code, so arbitrary self types will not help. As mentioned earlier, there is a certain magic which allows to call consuming methods on `Box`, but AFAIK it's currently not accessible to users. [Next page](https://internals.rust-lang.org/t/idea-introduce-core-read-owned/19038.md?page=2)