# Pre-Pre-RFC: Field offsets **URL:** \ **Category:** language design\ **Created:** [July 2, 2018, 12:14am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845 "2018-07-02T00:14:47Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mcy](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mcy/32/6512_2.png) [@mcy](https://internals.rust-lang.org/u/mcy)\ **Post date:** [July 2, 2018, 12:14am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/1 "2018-07-02T00:14:47Z")
I thought about the fact that there isn’t a good way to talk about offsets in a struct (i.e., C++'s pointer-to-member) while putting together my unnamables post. Some prior discussion can be found [here](https://internals.rust-lang.org/t/discussion-on-offset-of/7440). Currently, you can’t project fields through a custom reference type, like `Pin`. The workaround here is `unsafe { ptr.map(|x| x.field) }`, which requires `unsafe`, since the map function could move out of the `Pin`. To work around this problem, we introduce the type `T.U`, for `T, U` types. Then, we’d be able to write a function like this: ```rust fn project<'b, U>(this: &'b mut Pin<'a, T>, field: T.U) -> Pin<'b, U> { Pin { inner: this.inner.(field) } } // .. struct Foo { x: i32 } Pin::project(ptr, offsetof Foo.x) Pin::project(ptr, offsetof _.x) // with type inference! ``` The type `T.U` is a _field offset_ in the type `T` of type `U`, (i.e., an offset). This syntax is currently a parse error, so we can claim it for this use. We can use a field offset in place of a hardcoded identifier to access a field: ```rust struct Foo { x: i32 } let field: Foo.i32 = offsetof Foo.x; // take offset-of-field let foo: Foo = ..; foo.(field) // access the field pointed to by `field` ``` It is also acceptable to do a primitive cast to a `usize`, since this really is just an offset. While `Foo.x`, would be symmetrical with the field access `foo.x` and the type name `Foo.i32`, whether `Foo.x` is “offset of” or “access field” is ambiguous relative to whether `Foo` is a type or a binding. I’m using the `offsetof` as a placeholder until we come up with a better syntax. Alternative syntax: `.x`, though I don’t think the angle brackets are much better… The `.()` operator is used for the same reason, since having `foo.bar` mean different things depending on whether a binding `bar` is in scope is a bad idea. Given that we need to use parens in a strange way to call a member fnptr, I think this is acceptable (see: `(foo.f)(bar)`, since fields and methods are in different namespaces). The syntax isn’t amazing, but it’s better than the alternatives I thought of. Analogy with C++'s `T::*U` would be confusing (not to mention that it’s an awful sigil) since users might expect a `T::*mut U`, even though mutability is meaningless for a field offset. `T::&U` is worse, since it insinuates there’s a lifetime involved. Granted, I think that since this is a fairly advanced feature (I have _never_ encountered `T::*U` in C++ outside of a manual), it doesn’t need to be _supremely_ ergonomic. Edit: I wondered whether we could go further and overload field projection completely: ```rust trait Project { type Target<'a, T>; // requires GAT fn project(&'a self, offset: Self.T) -> Self::Target<'a, T>; } ``` However, C++ does not allow for overloading `operator.`, [for good reason.](http://www.stroustrup.com/bs_faq2.html#overload-dot) Unless we come up with a better way to drop down to vanilla projection than `unsafe { *(&foo as *const _).offset(offsetof Foo.x as isize) }`, this is a bad idea. I think this is about as much of a good idea as overloading `&` (which C++ allows!) We can probably do some clever things with the size of `T.U`, since structs very rarely have more than 255 fields. `T.U` will often be `u8`-sized, and, for structs with only one field of type `U`, a ZST. If there is no field of type `U`, `T.U` can be safely considered uninhabited! This allows for a rather silly function, to test if `T` has a field of type `U`: ```rust const fn has_field_of() { mem::size_of::() != 0 } ``` ---
**Author:** ![Centril](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/centril/32/3334_2.png) [@Centril](https://internals.rust-lang.org/u/Centril)\ **Post date:** [July 2, 2018, 6:50am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/2 "2018-07-02T06:50:30Z")
Interesting idea. Some miscellaneous notes: 1. `offsetof` is [unreserved in the beta compiler](https://play.rust-lang.org/?gist=cf9c464c18e38da687d4c638f16dfaf3&version=beta&mode=debug&edition=2015). The unreservation goes into stable on 2018-08-02. [Here](https://github.com/rust-lang/rfcs/pull/2421) is the RFC that unreserved it. 2. For `project`, it would be important that `T` is not polymorphic, which it isn’t in your snippet. Otherwise, you could leak private information about `Foo` which violates privacy (and thus leads to unsoundness). 3. The function `project` quantifies over `U` which is a `type`. This works well when all types in a type definition are disjoint. But what if we extend this to `struct Bar { x: u8, y: u8 }` ? Then we have to rely on the address to get the right place… Did you consider quantifying over field member names instead? 4. Continuing on 3. – would it / should it be possible to get the type of a field via `Type.field_name`? 5. Overloading field projection is interesting; but I think a central question is if it is overloaded for type constructors or for each type. In other words, are we extending it to `&'a` or to `&'a Foo`? If the former is the case, then we can do very interesting things given HKTs. ---
**Author:** ![mcy](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mcy/32/6512_2.png) [@mcy](https://internals.rust-lang.org/u/mcy)\ **Post date:** [July 2, 2018, 7:21am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/3 "2018-07-02T07:21:52Z")
> [@Centril](#): > > `offsetof` is [unreserved in the beta compiler](https://play.rust-lang.org/?gist=cf9c464c18e38da687d4c638f16dfaf3&version=beta&mode=debug&edition=2015). The unreservation goes into stable on 2018-08-02. [Here](https://github.com/rust-lang/rfcs/pull/2421) is the RFC that unreserved it. Indeed, I noticed that you're responsible for this. =P It's a placeholder syntax; I've also considered `.x` and `Foo.&x`, which aren't much better, but I'm open to ideas. > [@Centril](#): > > For `project` , it would be important that `T` is not polymorphic, which it isn’t in your snippet. Otherwise, you could leak private information about `Foo` which violates privacy (and thus leads to unsoundness). Elaborate? > [@Centril](#): > > The function `project` quantifies over `U` which is a `type` . This works well when all types in a type definition are disjoint. But what if we extend this to `struct Bar { x: u8, y: u8 }` ? Then we have to rely on the address to get the right place… Did you consider quantifying over field member names instead? Here's how imagine the sizes of an offset looking: let `T` and `U` be types. - If `T` is a struct, and there is exactly one field of type `U` in `T`, then `T.U` is an inhabited ZST, since we know at compile time exactly what offset it refers to. - If `T` is a struct and there a multiple such fields, then `T.U` is the smallest integer type that can hold the largest offset needed to access a field in `T` (almost always a `u8`... who the hell has 256+ field structs?!). - If `T` is a union, `T.U` is a ZST if `T` has a field of type `U` (reading `union.(field)` is `unsafe`, obviously). - ` [T; n].T` is as wide as the smallest integer type needed to hold `n`. (Unsure how we should allow for `[T; n].T` to be constructed... I think coercing from `usize` isn't an awful idea.) - In all other cases, `T.U` is uninhabited. I think that quantifying over the names of fields is... interesting, but I like the fact that it is possible to quantify over the type. Perhaps I could imagine giving each field of a struct a unique ZST, which coerces to a typed field offset. You certainly would not be able to use such name-specific types in a polymorphic context without adding identifiers to the list of things that can exist in angle brackets (no thanks, imo). > [@Centril](#): > > Continuing on 3. – would it / should it be possible to get the type of a field via `Type.field_name` ? Sure, this should be allowed. Continuing the analogy with fnptrs, you can totally do this via `FnTrait::Output`. Actually, it would probably make sense for `T.U` to implement `Fn(T) -> U`... Though recall that you can't name a field offset by `Type.field_name` due to the parsing ambiguity I mentioned. > [@Centril](#): > > Overloading field projection is interesting; but I think a central question is if it is overloaded for type constructors or for each type. In other words, are we extending it to `&'a` or to `&'a Foo` ? If the former is the case, then we can do very interesting things given HKTs. I don't quite understand what you're talking about here... I'd love an example! ---
**Author:** ![Centril](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/centril/32/3334_2.png) [@Centril](https://internals.rust-lang.org/u/Centril)\ **Post date:** [July 2, 2018, 8:03am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/4 "2018-07-02T08:03:17Z")
> [@mcy](#): > > Elaborate? Consider: 1. `Vec` in some module `A`. 2. `project` in some module `B` where `A != B` and defined as: ```rust fn project(this: &mut T, field: T.U) -> &mut U { this.(field) } ``` 3. `Vec` 4. In some module `C` where `C != A` ```rust fn main() { let mut vec = vec![0]; // If you forbid `offsetof Vec.len` here then all should be fine: let len = project(&mut vec, offsetof Vec.len); *len += 1; do_stuff(vec[1]); // Buffer overflow. } ``` Actually; given that you could make `Vec.len` not well-formed, maybe this isn't a problem. > [@mcy](#): > > Here’s how imagine the sizes of an offset looking: let `T` and `U` be types. Well reasoned 🙂 > [@mcy](#): > > In all other cases, `T.U` is uninhabited. This is a bit of a bummer. Does that not entail that we can't project to `Option`? Perhaps this can be solved with variant types. > [@mcy](#): > > Though recall that you can’t name a field offset by `Type.field_name` due to the parsing ambiguity I mentioned. Could be solved with: `Type.` ? > [@mcy](#): > > I don’t quite understand what you’re talking about here… I’d love an example! Very unbaked: ```rust trait Project { fn project<'a, T, U>(self: Self<'a, T>, offset: T.U) -> Self<'a, U>; } impl Project for |'b, T: type| &'b T { // Invented syntax for type lambdas... fn project<'a, T, U>(self: &'a T, offset: T.U) -> &'a T { magic } } impl Project for |'b, T: type| &'b mut T { fn project<'a, T, U>(self: &'a mut T, offset: T.U) -> &'a mut T { magic } } impl Project for |'b, T: type| Pin<'b, T> { fn project<'a, T, U>(self: Pin<'b, T>, offset: T.U) -> Pin<'a, T> { ... } } ... ``` ---
**Author:** ![mcy](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mcy/32/6512_2.png) [@mcy](https://internals.rust-lang.org/u/mcy)\ **Post date:** [July 2, 2018, 8:12am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/5 "2018-07-02T08:12:06Z")
> [@Centril](#): > > Actually; given that you could make `Vec.len` not well-formed, maybe this isn’t a problem. No way you'll get to make offsets of fields you can't see! The callee of `project` should rest assured that the caller has visibility for `field`. Formally: `offsetof T.x` will compile iff you can read `t.x` for `t: T` locally. If you do something like that to a type you own... well, your funeral. I should also point out that `Vec.len` is definitely ill-formed, because `Vec` does not have kind `*`. You want `Vec::.len`. > [@Centril](#): > > Very unbaked: Yeah I avoided HKTs in my example since GATs seem more developed. All you've done here is replaced my `Target` with `Self` which is... more constrained than I'd intended, but still gets the point across. I've also toyed with making `Project: Deref`, since overloading projection seems like something that should only be allowed with a smart pointer. `Project` doesn't use `Deref::Target` though, so it seems a bit artificial. I'm interested in what your thoughts are on the relationship between `Deref` and `Project`. ---
**Author:** ![Centril](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/centril/32/3334_2.png) [@Centril](https://internals.rust-lang.org/u/Centril)\ **Post date:** [July 2, 2018, 8:39am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/6 "2018-07-02T08:39:33Z")
> [@mcy](#): > > No way you’ll get to make offsets of fields you can’t see! Yeah; realized that after writing it down -- at least we agree on what should not be allowed now 😉 And it should be `Vec::.len`, agreed. > [@mcy](#): > > Yeah I avoided HKTs in my example since GATs seem more developed. All you’ve done here is replaced my `Target` with `Self` which is… more constrained than I’d intended, but still gets the point across. Not quite. There's also a difference in that you use `&'a self` while my variant permits you to implement it for `self` and `Constructor` as well. This is necessary if we want to capture partial moves of fields in the trait. Consider: ```rust struct Freeze { field: T } impl Project for |'b, T: type| Freeze { fn project<'a, T, U>(self: Self<'a, T>, offset: T.U) -> Self<'a, U> { self.field.(offset) } } ``` Or perhaps instead, using the family pattern and GATs: ```rust struct FreezeFamily; trait Project { type Source<'a, T>; type Target<'a, U>; fn project<'a, T, U>(src: Self::Source<'a, T>, offset: T.U) -> Self::Target<'a, U>; } impl Project for FreezeFamily { type Source<'a, T> = Freeze; type Target<'a, U> = Freeze; fn project<'a, T, U>(src: Self::Source<'a, T>, offset: T.U) -> Self::Target<'a, U> { src.field.(offset) } } ``` ---
**Author:** ![mcy](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mcy/32/6512_2.png) [@mcy](https://internals.rust-lang.org/u/mcy)\ **Post date:** [July 2, 2018, 9:01am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/7 "2018-07-02T09:01:08Z")
I will point out that the size of `T.U` does leak “private” information, insofar that I can tell if a struct as more than one field of a particular type by measuring `size_of::()`, but doing that would just be plain silly! ---
**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\ **Post date:** [March 25, 2019, 8:30am UTC](https://internals.rust-lang.org/t/pre-pre-rfc-field-offsets/7845/8 "2019-03-25T08:30:27Z")
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.