Problems with "dyn* Trait"

It's worth noting that typetag's purpose is to create a distributed slice of every tagged type, using inventory/ctor. I believe the referenced bridge trait is instead just the practice of turning

trait ObjectUnsafe {
    type Associated: Trait;
    fn do_something(&self) -> Self::Associated;
}

into the bridged

trait ObjectSafe {
    fn do_something(&self) -> Box<dyn Trait>;
}

impl<T: ObjectUnsafe> ObjectSafe for T {
    fn do_something(&self) -> Box<dyn Trait> {
        Box::new(<T as ObjectUnsafe>::do_something(self)
    }
}

manually as required.

3 Likes

(NOT A CONTRIBUTION)

Yes exactly, and bridges like this can get you very far. You can bridge both generic methods and methods using Self. The only thing they can't do are associated methods with no method receiver (since there's no way to bridge the dispatch without a method receiver). And they don't even require any unsafe code. This is in contrast to downcasting based on TypeId like Any.

But this technique seems not widely known. It should be possible to generate the bridge trait automatically. Or better yet, it should be possible to opt a trait into being object safe with the compiler generating the shims in the vtable for you so that you don't have the problems that arise from having two different traits.

3 Likes

Thanks for bringing up these questions, @withoutboats. I suspect we should fully see what the dyner crate does or some other proc-macro implementation of the feature does to properly answer some of these technical-related questions.

  • just as an example, &'r mut (dyn Trait + 'usability) is indeed officially non-covariant in 'usability, and yet a "reborrowing unsized coercion" is able to have that 'usability shrink. This is nevertheless sound since for dyn Trait to have a genuine API able to be given a value with a shrunk 'usability, then that lifetime parameter needs to be a generic parameter of the trait, which is then indeed non-covariant.

    But back to that remark, let's think of an interesting trait design:

    trait NonStaticAny {
        fn type_id(&self) -> TypeId where Self : 'static;
    }
    

    Then, let's consider a &'r (dyn NonStaticAny + 'usability): there is a huge difference between &'r (dyn NonStaticAny + 'static) β€”which is effectively a &'r dyn Anyβ€” and the others &'r dyn NonStatic Any (Playground).

    And yet dyn* NonStaticAny + '_ seems unable unable to capture that difference! So I totally agree with the statement that dyn* '_ loses precious informations, and that some (but not all) APIs would suffer from that.

My two cents would be that dyn* '_ + will, indeed, yield a more limited expressibility w.r.t. &'_ dyn, as the example above showcased, but I do suspect that it will be possible to define helper traits with the generic parameters inside the trait itself to palliate it.

  • e.g., it's easy to define trait Trait<'usability> : 'usability (e.g. trait MyAny<'usability> : 'usability), and then suddenly we have the "original 'usability" become visible in the trait itself, and, thus, the dyn* as well. Back to the previous example, we'd have trait Any = MyAny<'static>, and thus dyn* '_ + MyAny<'static> would be the way to express the difference back.

All that to say that it's not necessarily hopeless, and that I'd wait for a proc-macro polyfill of these things before judging the expressibility of these things, but I do suspect these kinds of "wrapper traits" will, at the very least, become more needed to palliate the lack of nuance in dyn* '_ + .

Being 'static has nothing to do with ownership: &'static str is "not owned", and Box<dyn Trait + 'lt> or just a Box<&'lt str>, like you said, is owned albeit non-'static. Ownership is exclusively about drop.

What the whole dyn* proposal is about, is having a VirtualPtr. That is, the whole drop logic of that type would be dyn, contrary to our current dyn Trait, which has mixed drop logic: part of it is statically dispatched. Indeed:

  • Box<dyn Trait> will statically dispatch a dealloc of the dynamically obtained Layout, as well as a dynamically dispatched drop_in_place of the pointee (hopefully not in that order :laughing:)

  • {A,}Rc<dyn Trait> will be similar but for tweaking the obtained Layout with the reference counters, and interacting with those reference counters in a statically dispatched manner (e.g., the whole dyn invocation of drop_in_place may be skipped).

  • &{,mut} dyn Trait will statically know that they need to involve no drop glue, and that &dyn Trait may be Copy, and that &mut dyn Trait may be reborrowed.

I think the properties of that last point are the main difference between an "owned" thing and a non-owned one.

In a dyn* 'usability + world (even when 'usability is 'static), there is then no difference between Box<dyn 'usability + Trait> and &'usability dyn Trait [1]: even if the latter were the original type which was (further) type erased to a dyn*, that information would have been lost, and we'd then still be involved with a non-Copy type since it would have a virtual destructor. That is, we'd have dyn ownership in and of itself.

  • In my previous "you don't own a &'static str", one could technically retort that in a way, we do own it, it's just that you have no drop glue to run / you can involve no-op drop glue. So back to this dyn* case, we implement "ownership" by offering a virtual destructor that would do nothing.

We could then envision a dyn clone operation, potentially not provided by every instance (hence fallible):

if let Some(extra) = dyn_star.try_clone() {
    …
}

and have &'_ dyn Trait offer it thanks to Copy, and similarly for {A,}Rc<dyn Trait>. Whether Box would offer it would depend on whether we implemented the dyn-safe workaround for Clone.

Regarding &mut-reborrowing, it would actually be the typical reborrow operation ("lending clone" of sorts :smile:) operation the we already feature for Pin<&mut _>, for instance):

trait Example {
    fn try_clone<'usability> (self: &'_ Self)
      -> Option<dyn* 'usability + Example>
    where
        Self : 'usability,
    ;

    fn try_reborrow_mut (self: &'_ mut Self)
      -> Option<dyn* '_ + Example>
    ;
}
  • And now that I see Option<dyn* …>, we then have other questions: quid of nullability (or rather, lack thereof)? I guess we'd have to forgo it since I guess that we'd want to be able to inline-coerce a usize into a dyn* Display, for instance, even if that usize is non-nullable. Or maybe the non-nullability of the vtable would suffice? :person_shrugging:

Conclusion

By virtue of type-erasing the very pointer type backing a dyn*, static / compile-time information is indeed lost:

  • yielding some cases of reduced API (lest we embed it back in the trait's non-erased generic parameters themselves).

  • and leading to dynamically dispatching / performing runtime-dependent (e.g., fallible!) operations such as try_clone, etc. for types that with our current dyn Trait design benefit from static knowedledge of the pointer type.

While type-erasing the instance itself makes a lot of sense for all the dyn case situations, I think most code does not need to type-erase the pointer type: it's something that most code knows in advance. It's only for APIs that want to be polymorphic in the pointer-to-dyn type itself β€”such as dyn-compatible traitsβ€” that the dyn erasure of that pointer type itself becomes a necessity: we get to remain polymorphic without sacrificing dyn-compatibility.

So it makes a lot of sense to have that proposal for dyn-compatible return-position-impl-trait-in-trait, but I have to admit that its usefulness elsewhere is yet to be seen.

  • I keep thinking, for instance, that &own "references" would be a better way of handling self: Self methods) for dyn Traits.

  1. This depends, of course, on Trait itself being &-transitive / on having a blanket impl Trait for &(impl Trait) / on Trait only having &self methods", etc., all of which has already been tackled by Niko's blog post. β†©οΈŽ

6 Likes

Ohai! I hadn't noticed this thread until someone pointed me at it (I've not been able to keep up with internals lately). Let me try to catch up and post some responses as I go.

I'm not following what you see as the issue here, maybe you can give a more complete example?

The idea is that one can construct a dyn* Trait + 'x from any type T where (a) T: 'x and (b) T: Trait. So, presuming there is an impl of Trait for &mut dyn Trait, then this should work fine -- the fact that in your original type you had multiple lifetimes is not particularly relevant.

This "hiding" of lifetimes is quite normal for dyn. You can do the same e.g. by creating a &'a mut dyn Debug from a &'a mut Vec<&'b u32>, which is totally legal.

Right now, we don't support quite the same hiding in impl Trait, but I believe we can (and the subtyping work I've been doing on a-mir-formality should allow us to express that cleanly, actually, since it adds more genreal existential types into the language).

I'm not sure how this would happen. The 'b in this case is a bound on the lifetimes that are hidden as part of the dyn type, and it cannot appear in your interface. If you had something like &'a mut dyn Trait<'b>, that would be a different story, the trait methods could reference 'b, but here 'b is being used as a bound on dyn.

2 Likes

Yes! The bridge trick is very cool. It does however have some limitations. In particular, if you have people define their own "dyn*-like" types, those types are (a) always tied to a particular trait and (b) not "canonical". So if I have a trait like this:

trait Timer {
   fn time(&Self) -> impl Future<Output = ...>;
}

and I want to make a "dyn variant" of Timer (let's say a struct DynTimer), I will presumably have to add some sort of annotation like #[derive(dyn)] that would define this DynTimer type (I probably have to specify the name for the type too; we can use a convention, that isn't always a great mix with hygiene, though).

But that derive is going to have to decide what kind of Future to return -- and it needs to be some sort of dynamically dispatched future type. So what does it use? We have to specify that too, since there is no one canonical "dyn future" type anymore. dyn* addresses this question.

We could of course have a "standard derive" that everybody uses, but at that point, we've kind of recreated dyn*. What's more, we have to solve some other questions like how to manage things that are potentially send or combinations of other trait, upcasting, etc.

I would definitely like to go in that direction, but I think in the end we are still going to want some default, easy to use option like dyn*. And dyn* is really quite general -- the only constraint that you have is that the underlying data has to be pointer-sized (and you have to have SOME fixed size or it just can't be compiled, so that limitation is inherent). It works with every kind of pointer type and can express a lot of bounds and protocols.

One other point is that I personally would love to ship something this year, and I don't see us shipping a complete, usable, and ergonomic solution if we have to start by adding (e.g) associated traits and other functionality (to accommodate things like dyn* Future + Send + Sync + Copy).

1 Like

It's true that we have fewer lifetime parameters, but as I wrote before, so does dyn. You have the same problem with dyn today by adding another layer of indirection.

Fundamentally the problem with this trait...

trait NonStaticAny {
    fn type_id(&self) -> TypeId where Self : 'static;
}

...is that it could not be implemented (or at least not used) for &impl NonStaticAny.

This is what I was getting at in the design when I talked about the need to have impls for pointer types: dyn* works well if you have such impls available (and indeed most code would like them to be available, it's annoying when they are not).

Right now, you can express things like &dyn Foo where &impl Foo doesn't implement the trait. That is a win in expressiveness for Foo, but I would argue it's kind of anti-pattern, and it's not very "object-friendly". In other words, dyn* asks for a bit more regularity from implementors but in turn makes a lot more traits work; dyn can accommodate edge cases but as a result works less well overall.

2 Likes

Ah, true, I totally missed that aspect!

So the goal of dyn* to erase the underlying pointer type, which is useful for many reasons. To do that you need to be able to speak in terms of what the pointer type can do rather than what it is. This is why I think "trait views" seem like the right abstraction.

The syntax has other benefits in that dyn* Foo can call all methods of Foo. This implies that it can only be created from a pinned pointer if any method requires Pin<&mut Self>, for instance. This means that instead of building up your capabilities by explicitly talking about Pin, you relax capabilities with &/&mut syntax in the same way that it works for most method signatures.[1]

You could imagine a world where dyn &Trait desugars to exists<T> dyn Deref<Target = T> where T: Trait, or something like that. An earlier version of this proposal took an approach that looked more like this. But now it takes a different approach, which is to remove Deref from the core feature entirely and offload that to the impl of the trait on the pointer type.[2]


  1. This raises questions about whether we also need syntax for dyn* pointers without the ability to call methods with other self types like Pin, Rc etc. I'm currently of the opinion that if you get into a situation like this you might just want to split your trait – these special self types are pretty uncommon, and in my experience most of them those are "core" methods of the trait you are calling, not auxiliary methods you could do without. But I could be convinced otherwise if there are examples where this isn't feasible. β†©οΈŽ

  2. There are still some implementation-level questions I want to work through with this approach. But it is elegant in the sense that it's easy to express and adds the ability to make it possible to use dyn* with pointer-sized types that aren't pointers. β†©οΈŽ

1 Like

I'd like to re bring up again my work on the storage API (current repo here); I believe it's possible to implement dyn* Trait as Box<dyn Trait, DynStorage>, and am actively working towards that goal.

Once I've finished proving out this approach, I plan to write an "inventing storages" blog post explaining why the API is the shape it is, and then I'd love to file a [MCP|RFC] as required to try out the storage API in-tree.

I think all I really need is a lang/libs team liaison to guide me through the process. I'm already tracking my progress on Zulip in #t-libs/wg-allocators.

This isn't to push the storage API as a "better" solution than dyn* (the language integration has its benefits), but just to make sure that it's known as an alternative.

8 Likes

For some reason I see a link, but when I click on it nothing happens.

One idea behind dyn* is to erase the pointer type so you don't have to hardcode your allocation strategy. If your proposal allows us to use Box without actually allocating on the heap, that's useful, though it wouldn't work for hiding an Rc. But there is some benefit to having Box represent ownership-by-reference and making that part of the type (hence proposals for &own and the like).

1 Like

Fixed the link, it was missing https:// so it didn't work :sweat_smile:

Box<dyn Trait, DynStorage> can support any heap allocation dyn* can in the same way (wrap the vtable). The Storage API is effectively putting the "inline or outline" feature into the type system, and you can already do vtable wrapping with (careful use of) newtypes.

(NOT A CONTRIBUTION)

My point is that, from your playpen link, make_debug1 cannot be expressed using this system. There is no equivalent signature to the return type with dyn*. Your proposal requires hiding that lifetime in all cases. A really trivial example is when you want to have a reference to a trait object but you want to constrain the trait object to be 'static. &'a dyn Foo + 'static doesn't strike me as a very exotic type signature for example.

(As a note, I cannot post proper code samples, so its hard for me to give more complete examples in general).

You're overpivoting on this specific case which matches async functions (returning an associated type you want to existentialize), but is not generalizable to all use cases. Serialize for example doesn't need to pick a pointer type for the Serializer it receives - the pointer type is reference. It just needs Serializer to become dyn safe (which is why it currently requires these extra traits in erased_serde).

As to the "fit inside a pointer" point, I'm confused about the semantics if it doesn't fit? If you're automatically boxing it for them, doesn't that mean that working in no_alloc depends on the size of the future type, which is likely an anonymous type? Good luck to your users figuring out if this async method will allocate or not! And if you don't do it automatically, how do they not allocate when not dynamically dispatched? They would have to say their future type is Box and always allocate.

I don't understand how the ?Send problem is alleviated by dyn* vs any other solution. I think your point has to do with derive syntax but that's not what I think is the solution to this problem.

I think the "default type" for this case should just be Box and let users override it when they want to opt into a different type like this small object optimizing box, which could be a library type.

Shipping something this year would be a remarkable change of pace from the past few years, but in my opinion too far in the other direction. You want a new way to do dynamic dispatch in stable within 8 months of the first blog post about it? That sounds incautious to me, even if I thought the idea was flawless.

Here's what I would do:

  • I would ASAP ship async methods without object safety.
  • I would thereafter ASAP ship object safe async methods without the ability to override the pointer type or things like Send, using very easy to understand and straightforward defaults (the ones used by async-trait), but know what syntax I want to allow specifying those things.
  • I would then introduce the ability to specify those things.
  • I would work on how to generate the bridge in the vtable generation so that users don't have to have bridge traits for a lot of cases like clone or serialize. This is intersects a bit with the previous points (e.g. picking the pointer type you want dynamic clone to evaluate to).
  • I would not pursue trying to hide the pointer type of a trait object at all.

And so what about when you want an Arc or Rc of a trait object? Or a Thin of a trait object? You want these for the pointer semantics and representation independent of the trait's methods.

Also, this means things like whether a dyn* type is copy or not depends on whether or not it has an &mut method in the trait body, or if you have a shared trait view? Right now, the system is simple: a shared reference is Copy and a mutable reference is not.

And what about APIs that take &impl Foo + ?Sized or &mut impl Foo + ?Sized. Will dyn* magically coerce into these generic pointer types, will it implement deref (see my previous note about how this means not getting rid of old-style trait objects)?

I don't think erasing the underlying pointer type is "useful" for any reason. What are the reasons that you mean?


We haven't even gotten into backwards compatibility hazards. When I add some state to a future, suddenly it needs to box to be dynamic, which means the performance implications of calling that method change and its no longer compatible with no_alloc. When I add an &mut method to a trait, suddenly you can't create a trait object of that trait from a shared reference and its no longer a Copy type. I'm sure there are more that don't pop up to me immediately.

The bottom line is this: the pointer type contains valuable semantic information that matters to users! Hiding it and trying to intelligently select the correct pointer type (with crazy small object optimizations to boot) for them based on non-local information like the trait and impl definitions leads to a world of pain and spooky action at a distance. This isn't like match ergonomics where the decision is just some local syntactic sugar based on immediately evident code patterns (and even with that people still complain). I don't think it will work well and it seems extremely out of line with Rust's design philosophy so far.

11 Likes

Niko's blog post doesn't go into this, but it wouldn't literally depend on the size of the type. Instead, the type would need to implement a trait like IntoSize:

trait IntoSize<const N> {
    type Raw: HasSize<N>;

    fn into_raw(self) -> Self::Raw;
    fn from_raw(Self::Raw) -> Self;
}

trait FromRefSize<const N>: IntoSize<N> {
    fn from_ref(raw: &Self::Raw) -> &Self;
}

trait FromRefMutSize<const N>: IntoSize<N> {
    fn from_ref(raw: &mut Self::Raw) -> &mut Self;
}

// Safe to derive, checked by compiler
unsafe trait HasSize<const N> {}

// Stabilize these first?
trait IntoPointerSize = IntoSize<size_of::<*const ()>>;
trait HasPointerSize = HasSize<size_of::<*const ()>>;

This sidesteps compatibility hazards by making types opt in to the trait. The implication is that the anonymous types you mention would not be returnable directly. We could make some special annotation on an async block/fn that allows a user to opt into it, but I wouldn't do this right away.

Speaking of annotations though, that is how a user controls the dynamic dispatch behavior. They can choose to box, or wrap the return value in another function (maybe identity) that returns a type implementing IntoPointerSize.

impl AsyncIterator for YieldingRangeIterator {
    type Item = u32;

    #[dyn(box)]
    async fn next(&mut self) { /* same as above */ }
}

impl<A, I> AsyncIterator for InAllocator<A, I>
where
    A: Allocator + Clone + HasSize<0>,
    I: AsyncIterator,
{
    type Item = u32;

    #[dyn(identity)]
    fn next(&mut self) -> Pin<Box<I::next, A>> {
        let future = self.iterator.next();
        Pin::from(Box::new_in(future, self.allocator.clone()))
    }
}

The effect is that we don't have silent compatibility or performance cliffs based on the size of your return value.

There is an earlier version of this proposal outlined here: Async fn in dyn trait. It uses a placeholder syntax dynx which is really the same thing as dyn*. It includes examples of all this, and shows how to implement wrapper types that override the dyn behavior of a type for a given trait. This allows preallocating return values on the stack, for example.

(NOT A CONTRIBUTION)

Thanks, I've read all of this link. Most of this seems great to me. My overall impression is that you all have dug really deep into this specific use case that async methods maps too: existentialising a return position type projection. The problem is that this doesn't generalise to all the other use cases for trait objects, so when you leave the realm of async methods and get into redesigning trait object types as Niko does in his blog post you end up with a solution that feels very magic and doesn't support all use cases.

But if "dynx" is just an anonymous type that dyn async methods return I don't really take issue with that. I'd suggest that there should be a way to concretize it to Box though (meaning requiring that the types for dyn(identity) need to support an API for boxing them) so its easier to use compatibly with other APIs that expect a Box<dyn Future>.

I'm a bit wary of this so-called dyner crate. You don't show the signature of InlineAsyncIterator::new anywhere & I don't know how you could write this function with Rust as it exists today, you would need a macro. I would want to iterate on this part a bit more; there's an obvious connection to the problem of stack pinning as well.

3 Likes

At first: currently we already have &dyn and &mut dyn kinds, they handle known cases of dynamic dispatch, and from them object safety rules arose;\

Given that the goal we are pursuing is allowing static existential types (impl Trait) to be coercible to dynamic existential types (dyn Trait) we just need to get "owning dyn types".

In Niko's blog post dyn* types are, essentially, fat pointer:

Creating a dyn*

To coerce a value of type T into a dyn* Trait , two constraints must be met:

  • The type T must be pointer-sized or smaller.
  • The type T must implement Trait

In the real world, however, what will actually be coerced to dyn* Trait is an owning smart pointer, such as Box<T,_> or StaticRc<1,1>. Thus, given vtable's, drops and custom allocators I think we should take a different approach: concrete layout of dyn* Trait type incorporates all these notions and we get a trait to construct dyn* Trait from appropriate types.

(Warning: I used generic traits concept a lot)

My preferred design is therefore:

/// this trait is implemented for everything that can be coerced into a `dyn*` type
/// this is unsafe, because the only types that should implement this must logically own their values (like `Box`)
unsafe trait CoerceDynStar<trait Trait>
   where Self: Deref, <Self as Deref>::Target: Trait
{
   //this is called on `dyn* Trait` value drop - it frees memory owned by the value
   fn drop(&mut self);
   //this is called for creating a `dyn*` value from a container
   fn create(self) -> dyn* Trait;
}
///`dyn *` types are, in fact, this:
struct DynStar<trait Trait>{
   data: Unique<[u8]> //owning data pointer (1)
   vtable: VTable<Trait> //vtable ref
   drop: fn(&mut self) //drop routine, shouldn't be a part of `vtable` field (see below) (2)
   allocator: &dyn Allocator //reference to an allocator which we got memory from*
}

* I know that currently allocator api isn't object safe (in old sense), but hope that this will change.

(1) This is the source of the lifetime 'x here dyn* Trait + 'x
(2) Moreover, this function pointer is only valid for the lifetime of allocator, which we have taken the memory from.

Thus, our API get more lifetimes:

//imaginary syntax warning

unsafe trait CoerceDynStar<trait Trait> 
   where Self: Deref, <Self as Deref>::Target: Trait
{
   fn drop(&mut self);
   fn create(self) -> dyn* Trait; //see below
}

struct DynStar<'d,'a: 'd,trait Trait>{
   data: Unique<[u8]> + 'd,
   vtable: VTable<Trait>,
   drop: fn(&mut self),
   allocator: &'a dyn Allocator
}

Since the structure exists only during 'd lifetime, not extending to 'a, we have only one lifetime we care about.

Given we are designing new kind of types I wonder if it'd be better if apply the same lifetime elision rules we have for output references to that single lifetime of dyn* kind? It'd allow writing (and transforming to ) fn method(&self) -> dyn* Trait instead of fn method(&self) -> dyn* Trait + '_

An example:

impl<O,'a,F: Future<Output = O>> CoerceDynStar<Future> for Box<F,&'a dyn Allocator> { //bad thing about generic trait happened
   fn drop(&mut self) {
     std::mem::drop_in_place(self);
   }
   fn create(self) -> dyn* Future<Output = O> { // (+ 'a) is unspecified
      DynStar::<'_,'_,Future>{
         data: self.ptr,
         vtable: VTable<Future>::for::<F>(), //this return vtable pointer\reference
         drop: <F as Drop>::drop,
         allocator: self.alloc, //alloc reference, hence attached lifetime
      }
   }
}

Dropping of any dyn* Trait type is done in two steps: call (self.drop)(self.data) and call self.alloc.dealloc(self.data).

P.S. Are there currently any cases that are not Box?

Side note: As I noted on Zulip, I've become convinced that dyn* Trait would be better expressed as Dyn<Trait> where we (eventually) move to supporting trait parameters like struct Dyn<trait T> { }. I'm going to keep using dyn* syntax for this post for consistency but I want you to know it pains me!

What I meant was that I want to ship async functions in (dyn) traits within a year, not necessarily a general dyn replacement. I don't believe shipping async functions is incautious; I think the problem is quite well understood. And yes, I expect we will start by stabilizing the static version, that seems like a sensible first step indeed.

I think your points about lifetime bounds are actually not the material point: the real key point is this. I agree that dyn* Foo (vs today's dyn Foo) means that the type no longer carries precise information about what kind of pointer (if indeed there is any pointer!) is being used -- that is indeed the whole purpose of dyn*. To stop having to talk precisely about which kind of pointer.

My argument is roughly this:

  • I think that vast, vast majority of use cases don't care which kind of pointer they have; they are forced to care today, but it actually adds speedbumps and makes dyn objects fit in less well.
  • There is no loss of expressiveness, because one can always capture whatever properties you are interested in via traits (I'm also assuming we generalize to arbitrary sets of traits). Most of the time, those traits will be something like dyn* Debug + Clone> instead of Arc<dyn Debug>, but if it really came down to it could do dyn* Debug + ArcLike where ArcLike lets you extract out the pointer / ref-count / whatever.

I think it's clear that there will be some code that reads better using today's dyn than with dyn* and various trait bounds. I think such code is relatively rare. I'd love to examine some of that code in more detail to get a better idea of what the various patterns are that people are using.

One related note is that I've believed for a long time it would be great to have a way to have "erased" type parameters. Previously I was thinking one might use the dyn keyword for that, but I'm going to instead talk here about a hypothetical erased just to avoid having too many variations on dyn (and because I don't think dyn would be the best choice if we adopted dyn*). In that case, one would be able to write code like:

fn foo<erased T: Debug>(data: &T)

I wonder how many of the use cases for things like Arc<dyn Debug> might be nicer to express this way; alternatively, one could imagine permitting Arc<erased Debug> as a "renamed" version of dyn that is useful in these situations.

2 Likes

The rest of your post makes sense, i'll have to pour over the details, but I'm confused by this question -- cases of what? Things from which we would construct a dyn*? If so, yes, lots! To start with, every pointer type :slight_smile:

2 Likes

You can see the prototype code here ---

the idea is that the crate would generate that on a "per-trait" basis (i.e., this is the output of the crate for the case of AsyncIter).

2 Likes

I read your post and I enjoyed it, but I'll have to dig a bit deeper -- but I think that the only way to make that truly work, if I'm not mistaken, is to support something like 'generic over trait bounds'. In particular, we want to be able to do dyn* Foo + Bar + Baz etc. (That said, I think we can support that, and it would be useful to do so.)

1 Like

(NOT A CONTRIBUTION)

Vehement agreement about this. It's the dyn* stuff that I'm concerned about, not async methods.

They're manifestations of the same underlying concern. dyn* doesn't express and compose in the way that the current system expresses and composes. This includes the distinct lifetime of the reference type, but also includes Rc and Arc, proposals for a Thin abstraction, Deref, DerefMut, Pin, Copy, possibly a type that could perform the small object optimization on arbitrary pointers, and so forth. By hiding the pointer, you lose the ability to compose trait objects with APIs that are designed to operate on arbitrary pointer types, and there are a lot of those.

I've already expressed my concern that in addition to being less expressive, this also introduces spooky action at a distance in which you have to look up the trait definition to figure out what the pointer semantics of a given trait object are. The idea that users just don't care if they have a Box or an Arc and a shared or a mutable reference doesn't ring true to me. Again I think this is overpivoting on the specific case that users want to return a trait object and they get confused about how to do that.

I think most of the speedbumps are either essential complexity or would be resolved if you just finished the features we've been committed to and apply very mild changes to the existing system to solve papercuts. I'm thinking impl Trait in traits, async methods, generators, automatically bridging for patterns like Clone and Serialize/Serializer. None of this requires dramatic migration to a new trait object system.

Finally, my sense is that you have failed to appreciate that the churn cost for Rust has grown by many orders of magnitude since 2018. I don't believe dramatic shifts like the one we did to the import system are very realistic anymore, and while they won't kill the language (because the language cannot die) they will cause a lot of pain to existing users and generate major reputational stink. The value proposition for a feature requiring such churn would need to be astronomical; even accepting your arguments this improvement seems incremental.

9 Likes