Revisit Orphan Rules

[gbutler]

The newtype workaround has lots of disadvantages which I mentioned in my original post aside from being inconvenient or tedious.

Could you clarify what disadvantages exist beyond inconvenient/tedious?

[mboratko]

First of all, I’m not sure why inconvenient/tedious is seen as less of a reason. Making code easier to read and write is one of the primary functions a programming language satisfies, and the newtype pattern adds unnecessary overhead for both.

Regardless, here is my post outlining issues which, even if you were handed a completed newtype pattern, are still disadvantages to using it:

Revisit Orphan Rules

The situation that sparked my question is what I’m offering as an example. Putting aside the tedious boilerplate issue, the main downsides to writing the newtype, in my case, are that: I would have to reimplement the functionality that derive(Serialize, Deserialize) provides manually, I can’t derive on a newtype because the internal types don’t already implement the traits. In my implementation I would have to manually specify the field names for the struct I was wrapping, which would make my …

[Ixrec]

Could you clarify what disadvantages exist beyond inconvenient/tedious?

I think we’re repeating ourselves now. I asked this same question earlier in the thread, and @mboratko pointed out things like code duplication making the newtype impl brittle, and my response was that I believe all those issues would also be solved by delegation, which is why I’d been lumping them together under “tedious boilerplate”.

First of all, I’m not sure why inconvenient/tedious is seen as less of a reason.

Assuming he’s coming from the same position that I am, it’s not “less of a reason” to change something, but it is a reason to think that the problem is not that the orphan rules exist, but that newtypes are too tedious to write and we need to figure out delegation.

[gbutler]

and the newtype pattern adds unnecessary overhead for both.

Here I think that the word “unnecessary” is your opinion and not at all clearly a fact. I again say, that what you are asking for is the equivalent of, “I have a class in C++ (or some other OO language) and I want to change the implementation of its base-class and have everything else use it as if it were the original without issue.”

I just don’t think that is a reasonable stance and I’ve seen nothing in all this commentary to convince me that it is a reasonable stance. So, the “unnecessary boiler-plate” you mention is entirely necessary glue-code to basically enhance a 3rd party class in a coherent fashion.

[elidupree]

I believe all those issues would also be solved by delegation

Don’t get me wrong, I think delegation would be great, but what about mboratko’s issue 4? Since nobody posted example code before, how about this:

//In crate foo
pub trait Trait {…}
pub fn do_something <T: Trait> (values: &[T]) {…}

//In crate bar
pub struct Struct {…}
pub fn get_values()->Vec<Struct> {…}

//In my crate
struct WrapperStruct (bar::Struct);
impl foo::Trait for WrapperStruct {…}
impl everything else for WrapperStruct by delegation;
let values = bar::get_values();
// now how do I do_something on values?

//compile error, structs haven't been wrapped
do_something (& values);

//Runtime overhead
do_something (& values.into_iter().map (| value | WrapperStruct (value)).collect::<Vec<_>>());

(Or maybe Vec in particular has some trick that allows it to optimize that series of operations down to nothing, but that’s not the point – clearly this issue would exist in many other cases.)

[Eh2406]

an example of an unimportant trate that would be great if everyone had but is not worth all the PR’s is https://github.com/servo/heapsize

[elidupree]

Okay, but that’s a clear-cut example of a trait that can’t be implemented legitimately by third parties.

[mboratko]

It can in the case of structs which have only public fields, no? (This is the situation I was in.)

[elidupree]

Ah, true.

I think the best language feature to address THAT situation would be a way to make blanket impls for structs with all public fields, similar to how you can currently make blanket impls for all tuples.

[illustrious-you]

First of all, I’m not sure why inconvenient/tedious is seen as less of a reason.

Assuming he’s coming from the same position that I am, it’s not “less of a reason” to change something, but it is a reason to think that the problem is not that the orphan rules exist, but that newtypes are too tedious to write and we need to figure out delegation.

Or, restating this in terms of @mboratko’s problem, we may be able to minimize the inconvenience using delegation, an RFC already under review. While that RFC does not solve the problem completely, it does reduce the scope of the problem to two cases:

• #[derive(...)] doesn’t work on newtypes unless the internal type also implements the trait, and
• changes to public fields can only be reliably discovered via code inspection.

As others have pointed out, the #[derive(...)] case is particular to crates like serde, which are relatively rare. Further, as noted by @withoutboats, changing the orphan rules to allow preferred impls will require recompilation, and wouldn’t solve the derive problem, because that would break type safety.

The other case, as I see it, comes with the territory. It may be worth considering whether a tool, such as a cargo plugin, could detect these changes, but I don’t see a way to build a feature into Rust without running into variations of the derive problem.

I’ve implemented a similar project in Python and Go in 1 day. It’s a standard “go to” project I use to try out programming languages.

It’s worth noting that both of these languages use runtimes that support reflection. This allows them to have more information about their types than Rust, which often eliminates types during compilation. Did you use reflection, or a feature that depends on it, to implement your project in Python or Go?

[mboratko]

Did you use reflection, or a feature that depends on it, to implement your project in Python or Go?

I’m not sure, I’d have to go back and inspect what I did.

Does Rust support reflection on external structs using macros during compile time? As I see it, that would be another way to handle this, although it feels like a bit of a kludge.

[gbutler]

Does Rust support reflection on external structs using macros during compile time?

If by external, you mean a struct defined in a different crate, then I believe the answer is no (please someone correct me if there is something I’m not aware of). There is an in-development run-time reflection API, rust-reflect. That being said, using run-time reflection to accomplish this goal, though imposing run-time overhead, might be the way to go for some of these kinds of things, but, it seems like a poor solution to the problem as well.

Perhaps, if there were a compile-time reflection API for structs that could be leveraged by procedure and/or rule-based macros, then many of these problems mentioned could be solved relatively painlessly. For example, if the answer is “New Type” wrapping to most of these problems, but, that is deemed too inconvenient/difficult to manage, then, could compile-time reflection coupled with appropriate macros provide the needed simplicity?

For example, in the Serde “Remote Derive” case, if Serde could define a macro that utilized compile-time reflection to create new-type wrapped remote- serde implementations for any 3rd party struct, would that resolve that case? Would other cases be amendable to a similar solution? NOTE: When I say “Compile-Time Reflection” I mean that there would be an annotation that could be placed on a struct (in the crate that defines the struct) that would cause compile-time reflection information to be generated and stored in the compiled lib. Then, macros, could have an API to access the stored compile-time reflection information about any struct. So, you could do something like

use SomeCrate::SomeModule::SomeStructThatHasReflection;

[#serde-remote-reflect]
type MySomeStructThatHasReflection : SomeTypeThatHasReflection;

To get a full “New Type” wrapped Serde Remote implementation for SomeTypeThatHasReflection provide it was annotated with the compile-time reflection annotation and that it met the rules that Serde would demand for auto-derive, like say, the struct must have no private members.

Maybe something like this, Reflect should be explored with respect to this problem space.

[arielb1]

1. I think that if we can find a nice way to have a “DieselChrono” crate without requiring Diesel to depend on Chrono or vice-versa, we’ll do a good amount on reducing the problems that come from the absence of coherence in practice.

   If such a crate can be allowed without Breaking The World terribly, I believe it will solve a real need in the ecosystem.

2. With specialization and Rust’s separate compilation model, breaking the orphan rules can have terribly confusing and very unsound (if associated type specialization is used) results, because a parent crate can observe an impl not existing while the child crate observes it existing.

   This should be mitigated when we get specialization_predicate to cut down on negative reasoning, as long as specialization predicates can’t be orphans.

3. For the “deriving Serialize on structs I don’t own” usecase, what you want is not being able to insert orphan impls, but rather to be able to reflect on fields of structs you don’t own.

   First, being able to impl Serialize yourself won’t be much help if you need to write the serialization code yourself. Second, if you can auto-derive, you don’t quite need to inject an orphan impl, but rather you can write your serde functions as free functions.

[CAD97]

If you have enough information to Serialize an external struct that isn’t Serialize, then you can use derive. Create a new type holding just that information, implement conversions, use the derive on it, and then use serde’s remote serialization feature.

Serialize is not motivation for this, as serde has a fully functional way of handling this, using the derive, through remote derive. Diesel/chrono is a much better example of where loosening the rules might help.

As a potential idea: what if you were allowed to declare an interop crate that was allowed to implement traits of each of two crates for the other. There would need to be some way to restrict it to one interop per compilation though. (We already have the issue with -sys crates though (of can’t have duplicate crates).)

[mboratko]

Create a new type holding just that information, implement conversions, use the derive on it, and then use serde’s remote serialization feature.

I really don’t understand why people seem satisfied with the additional performance overhead that this imposes.

[Ixrec]

Why do you think there’s a performance overhead to any of these suggested newtypes?

[gbutler]

As a potential idea: what if you were allowed to declare an interop crate that was allowed to implement traits of each of two crates for the other. There would need to be some way to restrict it to one interop per compilation though. (We already have the issue with -sys crates though (of can’t have duplicate crates).)

I mentioned something like this in an incredibly spit-bally fashion here: Moving bits of rustc into crates. I guess I could envision something like:

• Declare “Friends” Crates in the Cargo.toml
• Declare modules with “friend” visibility to “friend” crates in the main.rs/lib.rs
• Possibly have a “friend” visibility specifier for items to allow items that are private to a module/struct to be visible to “friends” of the module??? Otherwise, private items become effectively public to friends if not.

This probably requires more thought. Obviously, this would change the compiler as symbols that were private to the crate (or those with “friend” visibility specifier) would now have to be exposed to “friends” but not “public”.

Cargo and/or the compiler could possibly be taught to give warnings when changing “friend” visible items. Perhaps, Semver rules could be extended to say that any change to a “friend” visible item would require a minor version bump and that would be a breaking change for “friend” crates, but, not non-friend crates (or something like that).

[elidupree]

I thought some more about my last example, and I talked to a Haskell programmer about it. The central problem – converting a Container<T> to a Container<U> when you have a type T that is convertible to U – turns out to be exactly the use case for Coercible. I did some searching and found an old Rust RFC about this: https://github.com/rust-lang/rfcs/pull/91

Unfortunately, it hasn’t seen much attention since 2014.

Coercible seems like an ideal solution to a bunch of the inconveniences of the newtype pattern. With both delegation and Coercible, I think I’d find the newtype pattern convenient enough to use for this situation, reducing the need to relax the orphan rules.

[elidupree]

A couple things just occurred to me:

1. On one hand, Coercible potentially implements my idea of “a syntax for making blanket impls for structs with public fields” – if a struct with all public fields was Coercible into a tuple, then you could blanket-impl a trait for all types Coercible into tuples. Although maybe this would lead to too many overlapping impls – I haven’t thought out the details fully. Then you wouldn’t even need to explicitly use delegation in the newtype pattern, although…

2. On the other hand, for some traits, such as Ord, there are multiple reasonable implementations for the same type, and you might want to make a newtype that implements the same trait in a different way, or even doesn’t implement the trait at all. In conclusion, it seems important to make the newtype pattern convenient enough (through Coercible and delegation) that you can get away with the struct not already having an implementation of the trait. The “coerce into a tuple” idea is still useful this way, too, because when you’re making your newtype, it would let you delegate to the tuple implementation.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.