Custom DST Discussion

[nikomatsakis]

We recently postponed the custom DST RFC. But I wanted to continue the conversation both about its design and potential applications. If you’d like to catch up, read my most recent summary comment. Anyone, when we left it, @AtheMathmo and I were discussing trying to mock up how their library would look if implemented with this feature. I’m eager to hear the results. =)

[kornel]

The 2d slice of pixels is spot-on. I just refactored a bunch of functions taking width, height, stride separately into structs, and ended up with something that looks similar to 2D versions of Vec and [T].

Another related case I have is attaching color profiles to pixels (and palettes to 8-bit indexed pixels). If pixels and profiles are passed as two separate arguments, it’s too easy mix them up or create profile-ignorant/profile-erasing functions.

Having actual color profile pointer stored with every pixel is infeasible, so currently I solve it with Pixel(u8, PhantomData<Profile>) with types for some predefined profiles, but that only works for type safety. To have actual data related to the profiles (e.g. lookup tables) I still have to pass extra data around separately.

[mikeyhew]

I’ve been reading this whole thread of conversation, starting from the tracking issue for std::raw, then moving to the RFC on custom DSTs, and finally to here, because I had an idea about how std::raw::TraitObject could be replaced with something better. Something that lets you do everything you needed TraitObject for, that abstracts away the details of the underlying structure, and is a more general API that can be used on all types. Then I read @nikomatsakis’ comment on the RFC, which he linked to above, and realized he had the same idea.

Specifically I’m referring to the assemble and disassemble functions, that let you split a pointer into its (data_pointer, meta) parts, and put them back together again.

Here is an example of a problem whose solution could be much improved using assemble and disassemble. Yesterday I was creating struct DSTVec<T: ?Sized>, which is a generalization of Vec in that it allows types that are ?Sized. DSTVec would store a value’s data in a RawVec, and have a separate Vec of pointers into that RawVec.

Unfortunately, there was no way to take a *mut T and change where it points to. So I had to change tactics, and define a macro declare_dstvec! that takes the name of a trait and defines the macro for that trait. Here is a Gist.

Using assemble and disassemble, the implementation could be much improved. Here is a revision showing how it would work using the Referent trait. In fact, now that I think of it, I could probably create the Referent trait myself and use it to get rid of the macro!

[Fylwind]

I would like custom DSTs some day, though I wonder how much of the need is driven by the fact that the built-in reference type & and &mut are highly privileged in the compiler, i.e. there’s a lot of magic tricks that & and &mut can do that no other type can.

To name a few off the top of my head:

• References can be silently reborrowed
• Dereferenced &mut can be assigned to
• References can be (as)²ed into raw thin pointers
• Many built-in traits are exclusively reference-specific (Index and IndexMut, along with std::borrow and std::convert things)
• and probably many others I can’t think of because they are so ingrained into the language.

It would be nice if all the magic in built-in references could disentangled into independent aspects. One could then attempt to extended them to custom data types without bring in the full machinery.

After all, it’s conceivable that there might be a situation where you want some of the superpowers of references (e.g. reborrowing) without getting the other special abilities (like being assigned to, or even having a &mut counterpart in the first place).

[mikeyhew]

I just want to mention something that I realized recently.

@nikomatsakis if the Referent trait that you proposed in the RFC discussion is actually used in the compiler, then the assemble and disassemble methods shouldn’t be required methods. Their implementation would need to be defined by the compiler, because only the compiler knows the underlying structure of a *const T. And it could expose assemble and disassemble on the pointer types themselves, which makes more sense anyway.

[nikomatsakis]

if the Referent trait that you proposed in the RFC discussion is actually used in the compiler, then the assemble and disassemble methods shouldn’t be required methods

Yes, I see that in my comment I showed a sample impl where they were not provided – I agree that the compiler is responsible for synthesizing them. Probably this is just (ptr, meta), but anyhow.

[kennytm]

Just to sketch down the idea before it is lost.

In RFC 1309, one cannot use an OsStr as a pattern to split another OsStr because we could not cut a borrowed non-BMP character in half. However, if we represent the metadata of &OsStr on Windows instead as

{
    length: usize,
    extra_low_surrogate_before: u16,  // 0 for none
    extra_high_surrogate_after: u16,  // 0 for none
}

then I believe OsStr can be used as the pattern as well as the haystack for all “Pattern 2.0 APIs” there, instead of restricting the Pattern to UTF-8 input.

[nikomatsakis]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.