Rustdoc exposing private information

Rustdoc seems to expose unnecessary implementation details about private fields and #[non_exhaustive] in structs.

For example:

pub struct A0(());

pub struct A1((), ());

#[non_exhaustive]
pub struct A2;

#[non_exhaustive]
pub struct A3(());

#[non_exhaustive]
pub struct A4 {}

pub struct A5 {
    a: (),
}

#[non_exhaustive]
pub struct A6 {
    a: (),
}

I think all of these behave exactly same as far as their public API is concerned. And yet Rustdoc shows them all differently, exposing implementation details:

pub struct A0(_);

pub struct A1(_, _);

#[non_exhaustive]
pub struct A2;

#[non_exhaustive]
pub struct A3(_);

#[non_exhaustive]
pub struct A4 {}

pub struct A5 { /* private fields */ }

#[non_exhaustive]
pub struct A6 { /* private fields */ }

Similarly, I think these all behave identically from outside the module:

pub struct B0(pub i32, ());

pub struct B1(pub i32, (), ());

#[non_exhaustive]
pub struct B2(pub i32);

#[non_exhaustive]
pub struct B3(pub i32, ());

But Rustdoc shows them differently:

pub struct B0(pub i32, _);

pub struct B1(pub i32, _, _);

#[non_exhaustive]
pub struct B2(pub i32);

#[non_exhaustive]
pub struct B3(pub i32, _);

And also these:

pub struct C0 {
    pub a: i32,
    b: (),
}

#[non_exhaustive]
pub struct C1 {
    pub a: i32,
}

#[non_exhaustive]
pub struct C2 {
    pub a: i32,
    b: (),
}

Rustdoc:

pub struct C0 {
    pub a: i32,
    /* private fields */
}

#[non_exhaustive]
pub struct C1 {
    pub a: i32,
}

#[non_exhaustive]
pub struct C2 {
    pub a: i32,
    /* private fields */
}

Am I wrong, are there some subtle differences in the public APIs?

4 Likes

Anything in the crate can see past non_exhaustive, but only code in or below the module can see the default-level private fields. (Personally I find this disconnect from default privacy confusing.)

So non_exhaustive structs with no private fields can still be constructed elsewhere in the crate.


Is there value in not exposing the information? I'm more exasperated by having to click through to source when I happen to have a need to know than I am from having "too much" information exposed.

The error messages you get from trying to use the different versions also differ.

1 | pub struct A0(());
  |               -- a constructor is private if any of the fields is private

3 | pub struct A1((), ());
  |               ------ a constructor is private if any of the fields is private

6 |     let a2 = nex::A2;
  |                   ^^ private unit struct
  |

help: use struct literal syntax instead
   |
8  |     let a4 = nex::A4 {};
error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:8:14
  |
8 |     let a4 = nex::A4 {};
  |              ^^^^^^^^^^

error: cannot construct `A5` with struct literal syntax due to inaccessible fields
 --> src/main.rs:9:14
  |
9 |     let a5 = nex::A5 {};
  |              ^^^^^^^

error[E0639]: cannot create non-exhaustive struct using struct expression
  --> src/main.rs:10:14
   |
10 |     let a6 = nex::A6 {};
   |              ^^^^^^^^^^

error: cannot construct `A6` with struct literal syntax due to inaccessible fields
  --> src/main.rs:10:14
   |
10 |     let a6 = nex::A6 {};

Interesting. But in this context that distinction shouldn't matter because cargo doc doesn't document crate-private things.

There is a flag just for you!

$ cargo doc --document-private-items

If you run without the flag, that indicates you're not interested in the private details, so I think there is value in not exposing them.

Between C0, C1 and C2 I think the C0 version is the best:

pub struct C0 {
    pub a: i32,
    /* private fields */
}

Even if there are no private fields, the API says there might be, #[non_exhaustive] is morally the same thing as private fields, so I think it would be totally fine.

When I see this:

#[non_exhaustive]
pub struct C2 {
    pub a: i32,
    /* private fields */
}

I have to start to wonder what the docs are trying to tell me with this #[non_exhaustive] annotation before I realize they are not telling me anything useful.

Yes. I think those should also be unified.

4 Likes

Here is one way in which #[non_exhaustive] is different in intent that is relevant to documentation:

If I see a struct with only public fields and #[non_exhaustive], then I can know that the struct isn't currently hiding any state. This makes a substantial difference to how I can expect to use it — I know that any information that isn't in the public fields doesn't exist. Whereas, if it has private fields, I know that I will likely need to call functions to make full use of the struct, and there may be subtle distinctions between different instances.

Now, in theory, this is something that could be documented in the prose. But in practice, I've never seen a crate with documentation explaining this kind of thing — much as I wish there was documentation of things like "What state does this have? What possible distinctions are there between different values of this type?"

(Of course, the crate can always add fields in a future version, and they might as well be private as far as a dependent written against the past version knows.)


A different issue with rustdoc exposing information: I wish that authors could control whether #[repr] was exposed, because sometimes it's an implementation detail more than a promise to dependents. (Though in this case one can write a wrapper with a private field — and then the docs will say #[repr(transparent)] pub struct Foo(_), which is a little silly but not leaking information.)

2 Likes

Rustdoc includes the source code of the crate (can this even be turned off?), and basically all Rust crates are open-source (although I'm sure some people would love to change that). Given the status quo, you can never hide implementation details, you can only slightly obscure them. I don't see any issue with including them explicitly in Rustdoc. In fact, more often than not I'm annoyed that Rustdoc hides the private fields of structs and other private items, since I just need to dredge through source (which may consist of macros, and thus problematic to read).

3 Likes

Isn't the intent actually to prevent you from using this information? If it makes a difference to how you expect to use it, then you're using it wrong!

To me the intent is: it just so happens that there are no extra fields, but that's accidental, don't pay attention to that, it shouldn't be of importance to you and you shouldn't be relying on it (and it's likely soon going to change anyway in a minor revision).

Now I think by far cases A0-A6 are more common than B and C. It's just an "opaque type". It bothers me that it shows up differently in the docs depending on whether in the library it happens to be defined as:

pub struct Opaque(/* private */);

vs

pub struct Opaque {
    /* private */
}
1 Like

I think @kpreid is making a good point, let me try to rephrase it to make it more clear.

When a data type has private members, generally that means "there's some internal state you can't access directly, you need to use these specific methods to affect it, go read those methods' docs."

While often, #[non_exhaustive] means: "We might add a new field/variant in the future for new configuration options/new possible states/new diagnostic info/etc. But if you just want to produce one of the existing config options/set the state to one of the possibilities we support right now/read the diagnostic info we have currently, you can just access the fields directly and don't need to worry about methods and private fields."

1 Like

Oh I see, you're saying that in case there are only public fields, #[non_exhaustive] creates an expectation that future fields will also be public. OK. That kind of makes sense.

Although that's technically not implied by the attribute. Maybe there should be another attribute for that (#[non_exhaustive, forever_public]).

But I don't think it really makes a difference to how you use the struct in your code now -- regardless of whether they will be public or private, you can't support them now. They might as well already be there in the library, just kept private for now.

Maybe instead of /* private fields */, rustdoc should say something else in case there are any public fields, for instance:

pub struct Config {
    pub a: i32,
    pub b: i32,
    /* other */
}

I was really wondering something -- I thought that you could match the tuple struct as Opaque(..) despite it having private fields. This isn't the case, so why had I ingrained that using tuple structs locked you into being a tuple struct?

I can't know if that's the reason without a time machine, but as it turns out, it used to be the case that you could!

With rustc 1.16 and earlier, this compiles:

mod left {
    pub struct Tuple(());
}

pub mod right {
    use ::left::Tuple;    
    pub fn test(tuple: &Tuple) {
        match *tuple { Tuple(..) => {} }
    }
}

With rustc 1.17, that no longer compiles, and you have to match as Tuple { .. } instead, which was made stably available in 1.15.

I very concretely recall having used Rust some not insignificant amount before 1.16 introduced cargo check in March of 2017[1], so it's at least plausible... though how I just never ran into the error for matching private tuple struct fields in the past five years I don't know.


  1. Somehow doesn't feel like five years ago... and I think I was using Rust when core stabilized with Rust 1.6? My oldest GitHub activity with Rust is May 2017. ↩ī¸Ž

1 Like

That's the same feature request that @quinedot made, but it already exists:

$ cargo doc --document-private-items

I have no issue with that if what you want to look at is the implementation details. If you want to have a developer version of docs.rs that shows all the private fields that also makes sense.

But it makes no sense to go half way. Either you want to look at all the private fields, or you only want to see what is available in the public API.

1 Like

Given that a struct being a tuple struct and the number of fields used to be semver-public information way back when, I suspect the reason rustdoc doesn't hide that information is some combination of

  • It used to be important public information
  • It's not harmful to expose
  • #[non_exhaustive] is fairly recent, and basically just uses the preexisting functionality for showing semantic attributes rather than being handled specially
  • Normalizing the documented declaration is a not insignificant amount of work for negligible gain

In general, though, how rustdoc displays things is a lot more adhoc than the rest of the language tooling. A fun one I discovered in passing recently is that in some but not all positions rustdoc will replace dyn 'a + Trait with dyn Trait + 'a.

This was removed in this PR:

This is a fix to a privacy oversight that allows people to observe whether a tuple struct with a private field was defined as a tuple struct or not, something which was not intended to be public.

That's true but not what I meant. For a specific library version, when a struct's fields are public (whether or not they are non-exhaustive), I can read the documentation and know that the purpose it serves to me, a user of the library is fulfilled just using the data that it publicly has. I know that if none of the struct's fields contain some information I'm looking for, then I can't get that information using that struct — whereas if there are private fields, then there may be hidden information that I need to access through some use of a method (or function, or trait).

So, I don't want to claim that #[non_exhaustive] is especially useful information; rather that a struct not having private fields is useful information, and it'd be nice not to lose that fact.

To be clear, I'm not saying that inferring things from this kind of clue is good. I would much rather that every struct, whether it is non-exhaustive or not, whether it has private fields or not, clearly documents in some way or other what it does and doesn't semantically contain (from the perspective of API usage, not possibly-private fields). But, in practice, people often don't write that kind of documentation — they just tell you vaguely what the struct is about, and leave it to you to infer what that really means based on the available methods.

2 Likes

The RFC said the distinction between non_exhaustive and private / hidden fields should be explicit in rustdoc, to signal that more fields may be added.

Even when it doesn't reflect a difference in capability without changes to another crate, I don't really like the idea of errors becoming less precise by dint of crossing a crate boundary. A crate boundary isn't always some hermetic seal to the developer, especially in the face of workspaces. Sometimes you control both crates.

2 Likes

To some extent you got different error messages because your user code was different. You didn't show the code, but I think you wrote let a1 = nex::A1((), ());, let a4 = nex::A4 {};, etc, so it is no wonder that the error message are different.

I wrote this:

    let a0 = A0 {};
    let a1 = A1 {};
    let a2 = A2 {};
    let a3 = A3 {};
    let a4 = A4 {};
    let a5 = A5 {};
    let a6 = A6 {};

and got the following errors:

error: cannot construct `rust_test::A0` with struct literal syntax due to private fields                 
 --> src/main.rs:4:14                                                                                    
  |                                                 
4 |     let a0 = A0 {};                             
  |              ^^                                                                                      
  |                                                 
  = note: ... and other private field `0` that was not provided                                          
                                                                                                         
error: cannot construct `rust_test::A1` with struct literal syntax due to private fields                 
 --> src/main.rs:5:14                                                                                    
  |                                                                                                      
5 |     let a1 = A1 {};                             
  |              ^^                                                                                      
  |                                                 
  = note: ... and other private fields `0` and `1` that were not provided
                                                                                                         
error[E0639]: cannot create non-exhaustive struct using struct expression                                
 --> src/main.rs:6:14                               
  |                                                 
6 |     let a2 = A2 {};                             
  |              ^^^^^                              

error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:7:14                               
  |                                                 
7 |     let a3 = A3 {};                             
  |              ^^^^^                              

error: cannot construct `rust_test::A3` with struct literal syntax due to private fields
 --> src/main.rs:7:14                               
  |                                                 
7 |     let a3 = A3 {};                             
  |              ^^                                 
  |                                                 
  = note: ... and other private field `0` that was not provided

error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:8:14                               
  |                                                 
8 |     let a4 = A4 {};                             
  |              ^^^^^                              

error: cannot construct `rust_test::A5` with struct literal syntax due to private fields
 --> src/main.rs:9:14                               
  |                                                 
9 |     let a5 = A5 {};                             
  |              ^^                                 
  |                                                 
  = note: ... and other private field `a` that was not provided

error[E0639]: cannot create non-exhaustive struct using struct expression
  --> src/main.rs:10:14                             
   |                                                
10 |     let a6 = A6 {};                            
   |              ^^^^^                             

error: cannot construct `rust_test::A6` with struct literal syntax due to private fields
   |                                                
10 |     let a6 = A6 {};                            
   |              ^^                                
   |                                                
   = note: ... and other private field `a` that was not provided

There are only two different errors here, which makes sense if you accept the idea that non_exhaustive is different API from private fields. I still don't quite buy this, but I accept that there is perhaps a rationale for thinking of non_exhaustive differently.

In cases like A6 where there is both non-exhaustive and private fields, I don't think it's really necessary to talk about non-exhaustive because at this point the non-exhaustive really doesn't matter to you any more.

There are additional notes at the bottom about specific private field names. The notes are weird though, for instance it says: "and other private field a that was not provided" when actually it's the only private field. Why "and other"?

I think that note is misleading -- it suggests that I should be providing a when it's not going to work if I do. I think it would be better to simply tell me I cannot construct the struct because the definition is opaque, rather than telling me which private fields are missing.

Next I changed the code to try to construct with a specific private field:

    let a0 = A0 { a: () };
    let a1 = A1 { a: () };
    let a2 = A2 { a: () };
    let a3 = A3 { a: () };
    let a4 = A4 { a: () };
    let a5 = A5 { a: () };
    let a6 = A6 { a: () };

Now I get:

error[E0560]: struct `rust_test::A0` has no field named `a`
 --> src/main.rs:4:19
  |                   
4 |     let a0 = A0 { a: () };                
  |                   ^ field does not exist
  |                                                                                                      
 ::: /home/tomek/code/rust-test/src/lib.rs:1:12
  |                                                 
1 | pub struct A0(());                              
  |            -- `rust_test::A0` defined here
  |                                                                                                      
help: `rust_test::A0` is a tuple struct, use the appropriate syntax
  |
4 |     let a0 = rust_test::A0(/* fields */);
  |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~

error[E0560]: struct `rust_test::A1` has no field named `a`
 --> src/main.rs:5:19
  |
5 |     let a1 = A1 { a: () };
  |                   ^ field does not exist                                                             
  |
 ::: /home/tomek/code/rust-test/src/lib.rs:3:12                                                          
  |                    
3 | pub struct A1((), ());
  |            -- `rust_test::A1` defined here
  |                                                 
help: `rust_test::A1` is a tuple struct, use the appropriate syntax
  |                                                                                                      
5 |     let a1 = rust_test::A1(/* fields */);                                                            
  |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                             
                                                    
error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:6:14                               
  |                                                 
6 |     let a2 = A2 { a: () };
  |              ^^^^^^^^^^^^

error[E0560]: struct `rust_test::A2` has no field named `a`
 --> src/main.rs:6:19                               
  |                                                 
6 |     let a2 = A2 { a: () };
  |                   ^ `rust_test::A2` does not have this field

error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:7:14                               
  |                                                 
7 |     let a3 = A3 { a: () };
  |              ^^^^^^^^^^^^

 ::: /home/tomek/code/rust-test/src/lib.rs:9:12
  |                                                 
9 | pub struct A3(());                              
  |            -- `rust_test::A3` defined here
  |                                                 
help: `rust_test::A3` is a tuple struct, use the appropriate syntax
  |                                                 
7 |     let a3 = rust_test::A3(/* fields */);
  |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~

error[E0639]: cannot create non-exhaustive struct using struct expression
 --> src/main.rs:8:14                               
  |                                                 
8 |     let a4 = A4 { a: () };
  |              ^^^^^^^^^^^^

error[E0560]: struct `rust_test::A4` has no field named `a`
 --> src/main.rs:8:19                               
  |                                                 
8 |     let a4 = A4 { a: () };
  |                   ^ `rust_test::A4` does not have this field

error[E0639]: cannot create non-exhaustive struct using struct expression
  --> src/main.rs:10:14                             
   |                                                
10 |     let a6 = A6 { a: () };
   |              ^^^^^^^^^^^^

Still only two types of errors, but I didn't get an error for A5, I guess the compiler didn't get to that. If I comment out all the lines other than let a5 I get the third type of error:

error[E0451]: field `a` of struct `rust_test::A5` is private
 --> src/main.rs:9:19
  |
9 |     let a5 = A5 { a: () };
  |                   ^^^^^ private field

The notes at the bottom of errors for A0, A1 and A3 telling me to use tuple syntax are misleading and shouldn't be there. The compiler is advising me to use syntax that is not going to work!!

I think these errors would be a lot better if they all simply told me that I cannot use the struct expression at all because the type is opaque to me, rather than advising me about trying to use a different syntax (that will not work anyway) or telling me which private fields are missing (which again won't work).

In the case of A5 you could argue that maybe the mistake is in the other crate and the field shouldn't be private -- perhaps. Still I think it could be the same error message about trying to build an opaque type, with maybe an extra note at the bottom mentioning that perhaps the field in the other crate shouldn't be private (except if the other crate is std, in which case there is no point writing that).

Just for fun I tried something from std:

    let v = Vec::<i32> { len: 7, buf: [1, 2, 3] };
error[E0308]: mismatched types
 --> src/main.rs:4:39
  |
4 |     let v = Vec::<i32> { len: 7, buf: [1, 2, 3] };
  |                                       ^^^^^^^^^ expected struct `alloc::raw_vec::RawVec`, found array `[{integer}; 3]`
  |
  = note: expected struct `alloc::raw_vec::RawVec<i32>`
              found array `[{integer}; 3]`

Wouldn't it be a better error message if it concentrated on the more important fact that I'm using a struct expression for an opaque type, rather than going into nitty-gritty details of private fields in a different crate?

It could be something like:

error: cannot use a struct expression for an opaque struct `Vec`
 --> src/main.rs:4:39
  |
4 |     let v = Vec::<i32> { len: 7, buf: [1, 2, 3] };
  |             ^^^^^^^^^^^^ trying to construct an opaque struct
7 Likes

The code for the first set of errors was

    let a0 = nex::A0;
    let a1 = nex::A1;
    let a4 = nex::A4;

(using all 7 variations, but I only included unique error examples), and for the second set of errors was for

    let a4 = nex::A4 {};
    let a5 = nex::A5 {};
    let a6 = nex::A6 {};

I.e. the same code gave different errors depending on the struct particulars.

Emphasizing that no matter what, literal struct syntax isn't going to work here, definitely makes sense. Perhaps by expanding E0451 to explain that you can't use struct literal syntax for construction or FRU and what causes it (private fields or non_exhaustive).

There's a history of diagnostics for this case (you didn't supply everything, but you can't anyway):

I still think it should still be emitting actual E0451 errors like it does when you supply all the fields. But yes, the emphasis should be "this just isn't going to work, look for a constructor function or something."

The output where you supply all fields hasn't changed in some time, so apparently it's in a different diagnostic path -- it should also emphasize that literal struct syntax is not an option.

I agree with that, and I think it should be the primary message. But I also think there's value in pointing out that a particular field you already provided was private. Apparently you knew what the field was somehow, just not that you couldn't initiate it.

5 Likes

Yes with #![doc(html_no_source)].

2 Likes