Revisit Orphan Rules

[ExpHP]

Because afaik that’s not the case in eg. Haskell.

That’s because Haskell has a runtime.

[Ixrec]

Ah, fair enough. I was lumping in the duplication part with the “pure boilerplate” that will never have to change because “delegation” should solve both.

As for the performance overhead on 4 and 5, are you sure that wrapping/unwrapping a type nested within another type (say a vector) is a noop at runtime? Because afaik that’s not the case in eg. Haskell.

I think we’re gonna need a concrete example here. There’s certainly no reason why x.y.foo() would be any more expensive at runtime than y.foo() if x is just a newtype around y; typically x would vanish during optimization.

[mboratko]

Ah, fair enough. I was lumping in the duplication part with the “pure boilerplate” that will never have to change because “delegation” should solve both.

Yes, put more simply, from my perspective this list (aside from 6) are all downsides to using an already completed newtype pattern.

3 and the rest of 2 are basically the dilemma we’ve mentioned many times before that you often want serializations to be deserializable into future versions of the same type, which is impossible to guarantee unless the type’s owner provides the (de)serialize impls

That’s not the issue I’m pointing out here. For my implementation, I want lightweight clients that just capture the raw structs and pass them over the wire to a server, which will dice them up and save them in a different format. I probably won’t want to use all the fields on the server, but I don’t want that logic spread out between the client and the server. If I decide that I want to collect more of the fields, I would have that capability solely by updating the server.

If I was just able to derive Serialize, Deserialize on these structs my program would only be dependent on the eventual public fields that I used, as opposed to making a newtype and then copying over every single field in the existing struct, resulting in a complete dependence on the full external interface of the struct, which is what I mean by “brittle”. In particular, in this situation I wouldn’t particularly care about backward capability of the deserialize method, as long as I updated the clients and the server at the same time everything would continue to work even if the struct author had added new fields I was unaware of.

[Ixrec]

Hm, okay, I did misinterpret that, but your actual issue still sounds like something that’s unique to the Serialize trait. Is that fair?

[mboratko]

I don’t think so.

For example, what if there was a Loggable trait. The reason for it’s existence is that instead of just using Display or Debug it allows you to pass it a struct and a verbosity level, and some fields would be hidden under a particular verbosity. Maybe the default derive implementation of this trait is to have the verbosity = level of nesting in the struct, but obviously some structs may implement this on their own. In addition, there might be formatting parameters for different logging conventions, and the information might be different than what you’d get from Display or Debug, for example maybe it also includes information about the size of the struct or the time, the goal being to be able to write something like

x.log(3);  // sends x to the current logging mechanisms with verbosity 3

[Ixrec]

I’ve been trying to think about counterintuitive cases that would arise from this, but it’s surprisingly difficult.

iiuc the problems with allowing orphan impls are less about causing code to do counterintuitive things at runtime, and more about enabling a non-trivial dependency graph of crates to code itself into a corner where the code I want to write is literally impossible to express because of conflicting impls (and conflicting ad hoc resolutions to deal with other conflicting impls) provided by various dependencies.

Following up on my previous post, it gets worse if you think about supertraits:

Though this is a good example of how little it takes to make it confusing. I think this is essentially the same thing as the “hashtable problem”, albeit as user confusion rather than unsoundness or incorrectness.

[Ixrec]

That’s basically what I was talking about in this previous post. Perhaps we’re talking in circles at this point.

[mboratko]

So, to be clear, you’re saying these sorts of traits are unique because they are the only ones where all the fields might be necessary? Maybe in practice that’s true, honestly I don’t have enough practical experience to offer an opinion. I see your point though, it’s hard for me to imagine other traits which might have to arbitrarily recurse through the entirety of a type’s structure.

[Ixrec]

I do think (De)Serialize are the only traits/interfaces I’ve ever encountered that always need to know about all the fields in a type in order to be implemented correctly.

There are traits that usually need to know about all or most of the fields in a type, but I believe those are mostly the Debug/Eq/Ord/etc traits in std, and that’s why they’re also on the “always implement if possible” list.

afaik, it’s extremely rare for any other trait implementation to need to operate on the type on a “field by field basis” to achieve basic correctness, and just about every other trait can get away with the type’s public API (if it makes any sense to implement that trait for that type). Maybe I’m missing something important, but that is the impression I get from the current state of Rust and all these other languages we’ve mentioned.

---

So, personally, I think what we really need to make progress on this issue is more concrete examples of very specific use cases not involving Serialize where “just use a newtype” didn’t work.

The only one I’ve heard of personally was a comment (maybe by @sgrif?) that diesel-chrono should be a thing that exists, but it wouldn’t make sense for diesel or chrono to depend on each other in order to provide those impls. If it turns out that all use cases are similar to this one, perhaps we could brainstorm some sort of whitelisting mechanism where diesel and chrono both declare that diesel-chrono has special permission to impl their items.

[elidupree]

afaik, it’s extremely rare for any other trait implementation to need to operate on the type on a “field by field basis” to achieve basic correctness

How about a trait for a tracing garbage collector?

I’ve also run into at least one other case where I had a custom type, let’s call it WeirdStruct, and I wanted to implement an algorithm that takes an object and visits all instances of WeirdStruct owned by that object. In my current code, I’ve implemented this using the horrible hack of piggybacking off existing Serialize implementations, but if I wanted to release something like that in a library properly, I’d have to depend on everyone else to implement a VisitOwnedWeirdStructs trait.

[withoutboats]

Even if we did allow users to violate the orphan rules, they’d have to use the public API of the types to write the orphan impls. We could not just let you derive Serialize for a third party type and visit all of its fields. Libraries are allow to assume that other users cannot safely access the fields of their types, and depend on that for safety.

[gbutler]

I honestly do not understand this thread/discussion. It feels like an absurd proposition on its face and I haven’t heard any rationale for why it would make sense or how it would work and be coherent. All I see/hear when I read this is the equivalent of, “I want to change the base-class/interfaces implemented by some class I did not implement and do not control.” I think this coming from the idea that Rust is like a dynamic language where you can just change prototypes and such and everything just “works” (for some definition of “works”). I just don’t see, and nothing in this discussion has made me feel moreso, how this is feasible and/or useful for Rust.

If those advocating this are serious, they are going to need to produce some real concrete examples that actually make sense.

[mboratko]

Of course, I’m not suggesting otherwise.

[mboratko]

What about the example I gave already?

Subtypes are one way to make this coherent. There might be implementation issues with doing so, but it’s not impossible.

[gbutler]

They are unconvincing (at least to me). Perhaps, like, @Ixrec said, diesel-chrono would be a better example to discuss and “spit-ball” about. Just not seeing the point WRT serde.

[elidupree]

I honestly do not understand this thread/discussion. It feels like an absurd proposition on its face and I haven’t heard any rationale for why it would make sense or how it would work and be coherent. All I see/hear when I read this is the equivalent of, “I want to change the base-class/interfaces implemented by some class I did not implement and do not control.” I think this coming from the idea that Rust is like a dynamic language where you can just change prototypes and such and everything just “works” (for some definition of “works”). I just don’t see, and nothing in this discussion has made me feel moreso, how this is feasible and/or useful for Rust.

If those advocating this are serious, they are going to need to produce some real concrete examples that actually make sense.

Not commenting on any specific proposal, but I can definitely give examples where it would make sense. Let’s say a library provided the trait VisitOwnedWeirdStructs from my last post, but only implemented it for std types. I’m pretty sure I could implement any of these, if not for the orphan rules:

impl <T: VisitOwnedWeirdStructs> VisitOwnedWeirdStructs for smallvec::SmallVec <T> {…}
impl <T: VisitOwnedWeirdStructs> VisitOwnedWeirdStructs for arrayvec::ArrayVec <T> {…}
impl <T: VisitOwnedWeirdStructs> VisitOwnedWeirdStructs for nalgebra::Vector2 <T> {…}
impl <T: VisitOwnedWeirdStructs, U: VisitOwnedWeirdStructs> VisitOwnedWeirdStructs for splay_tree::SplayMap <T, U> {…}

Because I know they are container types that only contain the values I put in them, and they expose all of those values through an interface.

If there were general traits for containers that can be iterated over, it might be possible for the trait library to create blanket impls that would cover these cases, but there aren’t, so there isn’t really a way for the library to provide these impls (other than by manually implementing it for each container library the author knows about).

[gbutler]

And why wouldn’t the NewType pattern suffice here?

[sgrif]

perhaps we could brainstorm some sort of whitelisting mechanism where diesel and chrono both declare that diesel-chrono has special permission to impl their items.

That kinda defeats the purpose. The idea is that someone should be able to make this crate without either of the parent crates having to know/care/get involved

[mboratko]

They are unconvincing (at least to me).

Why not?

To give more context, In my specific example, I’m trying to make a very simple monitoring app. I want to collect stats from the clients and send them back to the server. I want to push out the client apps once and have them send all their stats back to the server, where I will then decide what to save. I want to be able to adjust what data the server saves without updating all the clients. (Sending the whole struct, as opposed to creating a new struct and selectively choosing the data to send, is also useful for prototyping reasons.)

As it currently stands, I am not satisfied with the newtype pattern for all the reasons above, and thus I’ve forked the systemstat crate and derived Serialize, Deserialize but I’m waiting on a dependency of that crate to push it’s new version which implements Serialize, Deserialize to crates.io in order to then submit a pull request to the existing crate, which I will then wait again for…

I’ve implemented a similar project in Python and Go in 1 day. It’s a standard “go to” project I use to try out programming languages.

I’m not trying to “port” my existing implementations, I’m really interested in learning the idiomatic way to do this in Rust, but so far it’s been a no go. If I could have derived these traits myself I would have been done already.

[gbutler]

and they expose all of those values through an interface.

Hmmm…this might be an interesting point to latch onto? Perhaps, what is wanted is the ability to to provide 3rd party impl of traits in terms only of other traits without direct reference to particular structs?