[lang-team-minutes] private-in-public rules

I am uncomfortable with this -- it seems to me that, for consistency, it makes sense to enforce the rules at the impl site, particularly if -- as you say -- we need to enforce privacy at the point of pub use. That is, the impl site is analogous to a pub use, where the alias is being created through the trait system, instead of the module system.

I contend that quite a bit is being leaked, even if you are restricted to impl Trait style matching. It seems surprising to me that instances of my private types can be created and manipulated without my code ever being involved at all. Granted, the public code only gets access to the APIs available through public traits, but that can be quite an extensive API surface.

Is there a type T which precludes &T : Copy?

Oh, this is unfortunate :frowning2: I assumed the privacy issue was the only roadblock for some reason. This makes disabling privacy checks for predicates less urgent.

I donā€™t think checking associated types early improves consistency (quite otherwise), because use and associated types are sufficiently different in details. I also donā€™t think giving away private types in anonymized form is so bad - generic code in the standard library works with my private types all the time! I wonā€™t argue however, because early checking of type aliases is more conservative, future compatible and doesnā€™t require any extra work :slight_smile: Iā€™m also not sure itā€™s an important case in practice. It can always be reconsidered later if desired.

I spent some time thinking about this thread this morning, and have a few scattered observations. Some of these are repeats from the thread but I want to summarize.

  • If a value of a given type is possessed by a piece of code, that code can invoke any visible API (e.g. Drop)

    • Even types returned via impl Trait have this property, due to specialization.
    • Trait objects and newtypes seal up the API exported, but always include Drop.
  • The visibility of a type has no inherent importance; what matters is what APIs can be accessed by which code.

    • Making a type private guarantees at most that external code can only see its public APIs. Thatā€™s because code within the module can pass values of that type to generic functions.
      • As @nikomatsakis points out, you canā€™t event guarantee that code is executed within the dynamic extent of the module.
    • @nikomatsakis argues that thereā€™s a difference in spirit between passing a value to a generic function and having it as a return value or associated type, but Iā€™m not sure I agree.
      • In both cases, the module author takes a type from having no APIs accessible from outside to having all public APIs accessible.
  • With pub(restricted), we talked about a pattern to improve ergonomics:
pub(crate) struct Foo {
    pub field1: Type1,
    pub field2: Type2,
    ...
}

Here we (lazily) use pub on the fields, knowing that the struct itself has limited visibility, and therefore the fields are effectively capped at crate-level visibility. The reliability of this trick will depend on what kind of escaping we allow. But it might be possible to make this trick work by re-interpreting visibility annotations so that visibility is the intersection of all inherited restrictions, or some such.


Iā€™m not sure what conclusions, if any, to draw from the above, but I think the point that marking a type private can give you only limited guarantees is an important one. I noticed some inconsistencies in argumentation around this point, where sometimes weā€™re arguing in favor of ā€œlocal reasoningā€, but in other places we note that such local reasoning is effectively impossible.

Could you elaborate on this point? With examples of leaking private things through impl Trait + specialization, if possible. I remember I read something about specialization being able to reveal types anonymized by impl Trait, but don't remember details. (If interaction of impl Trait and specialization breaks otherwise sound privacy system, maybe it's this interaction that needs to be fixed and not privacy.)

Also this. What are these other places?

Note that you can de-anonymize impl Trait using Any, even without specialization.

@apasel422 gave an example, but to elaborate slightly: there's a tradeoff here between the level of sealing we want to provide for impl Trait and performance gains via specialization. I think this tradeoff is basically orthogonal to other points about the privacy system, since calling a generic method with a private type is already enough to expose all public APIs anyway.

Sorry, I wasn't being clear. The core of what I'm trying to say is the following:

  • The best local guarantee we can provide about a private type is that external code can only access its public API.

That's because of the potential for code within the module to invoke generic methods with values of that private type.

This "guarantee" doesn't seem to provide very much; after all, private APIs are inaccessible regardless of whether the type is public.

So I think that arguments based on the ability to draw conclusions from the visibility of a type alone don't carry water. That seems significant, because we've tended to put a lot of weight on the ability to "reason locally" based on these declarations. I think we can reason locally about the visibility of APIs (like fields and traits), but that type visibility just never tells us that much, locally.

Now, where those thoughts lead in terms of the design, I'm not sure yet...

I don't fully agree with this formulation. If that were the whole story, then we wouldn't bother with any guarantees, right? It'd be perfectly fine to "infer" the type of a local variable to a private type, e.g. in code like this:

// here `returns_some_private_type()` returns a type
// not visible to the current module... we couldn't write
// the type explicitly, but we can *infer* it.
let x = foo.returns_some_private_type();

// Regardless, we are forbidden from accessing private
// methods. We are limited to the public API surface,
// though notably this includes all traits.
x.public_method()

If we adopt the more liberal rules advocated by @petrochenkov (namely, it becomes illegal to have a type that is inferred or normalized to a private type, but it is ok if the type name is not "visible" statically because it can't be inferred or normalized), then the code above becomes illegal, and the guarantees around private types become something more like:

"A private type can only be used from outside the current module via trait methods"

Even here it is not possible, in the strictest sense, to "detect" the type via specialization, because one cannot write the specialized impl since you can't name the type directly. It may be possible to write some indirect impls via associated types, but they will still be limited to trait methods.

If we adopt the slightly stricter version that I proposed (in which we retain a rule prohibiting associated types from being assigned to private types, unless the impl has a private input type), then we get an additional guarantee, which has to do with the origin of the private type. In particular, "original" instances of the private type can only be created within the owning module. They can then escape the module's dynamic scope via returns, threads, or mutation.

So, as a simple example, imagine a private Priv which implements Default:

trait Alias {
    type Out;
}

mod priv {
    #[derive(Default)]
    struct Priv;
    pub struct Pub;
    impl Alias for Pub { type Out = Priv; } // illegal under my proposed rules
}

mod another {
    // directly naming the type via this alias is illegal under any proposed rules:
    type Priv = <Pub as Alias>::Out;

    pub fn foo<T: Alias> {
        // but indirectly naming it is ok, and now I can create an instance:
        let x: T::Out = T::Out::default();
    }
}

fn main() {
    another::foo::<priv::Pub>();
}

It seems to me that having control over when values are created is really important, particularly when you have types that represent permissions. That said, probably those types would not implement Default or any similar trait that lets you synthesize said permissions from "thin air". But I'm not entirely sure about that.

It is also certainly true that the story of said guarantees is decidedly not simple. Your approximation (public API surface is accessible) is a good rule of thumb, though it overstates the truth.


I admit, the analogy between pub use and an impl didn't occur to me at first. But now that I've seen it I can't unsee it. =)

That said, I think that ultimately we're going back and forth on fairly fine point, and by and large the guarantees you get are very close either way. It comes down to whether we think it's ok that people can synthesize instances of private types "from thin air" outside of the module.

1 Like

It seems we crossed wires, but my quibble is that the various kinds of privacy rules proposed do prevent "inherent" member access (fields, methods). They do not prevent access through traits though.

Just saying that it is an error to access private methods/fields/etc (but allowing private types to be inferred elsewhere) feels like a relatively simple rule, but it does admit a few odd things:

  • It is odd that there are types which it is ok to infer but which, if typed explicitly, would yield an error.
    • This is why we want to enforce errors if inference leads to private types.
  • It feels odd (to me) to have errors for types, but only if the code is written generic and not 'hand monomorphized'.
    • This is sort of my example from the previous message.
  • Similarly, it feels odd (to me) to have to normalize an associated type to find out if a type is illegal.
    • I would prefer a version of the type system in which normalization is only needed if we have to prove equality to something, not for its own sake.
    • Certainly my new trait system prototype has been following this principle. =)
    • I admit this is more of an "abstract" concern.

And then there's the business about having "control" over creating instances of your private types. But I have a hard time...formalizing this one. I mean, even in my Default example, the Default::default method was defined in the priv crate, so in some sense, code from priv did execute. But it feels very analogous to calling an inherent pub fn foo() -> Priv, which would be illegal, except that it is legal to call <T as Default>::default() where T winds up being Pub.

It seems like I confused myself in this list.

This would occur with the most lenient rules.

This would occur with @petrochenkov's variant (as would the next)

For the line of argument I'm trying to make, I think it's important to distinguish between type visibility and API visibility. But it's true that there's a question around code like:

struct Private;

impl Private {
    pub fn foo(&self) { ... }
}

Namely, if you get a hold of an instance of Private in a context where you can't access the type, are you able to invoke foo? I think this is pretty similar to the issue I was raising about the pub(restricted) hack for ergonomic privacy constraints.

Regarding the oddities you mention, I don't disagree, and they all feel somewhat akin to the question of publicly re-exporting a private type. But what I was mainly trying to get at was the "local reasoning" guarantees we can hope for with private types. In particular:

While I agree that the module does control instance creation, my point is just that it's a non-local consideration, in the sense that you can't tell whether/where it's happening just by looking at the type definition. (It's possible there are multiple degrees of local reasoning we should be thinking about; I've been focusing on what conclusions we can draw from type/API definitions alone.)

Going back to our original thinking for the privacy system, to me the main motivation for stringent rules is to support meaningful local reasoning based purely on definitions. If the guarantees are not so meaningful, and the motivation is more about reducing oddities, then it feels more like a lint to me.

Oh, one more thing about this

I'd like to again draw your attention to my comment about link time visibility because this is one of two most important things I want from privacy system (another one is ability of silly humans to reason about their code). From that point of view "external code" is "code in crates depending on our crate" and "its public API <for private type T>" is ideally "impls of public traits for T participating in impl Trait and nothing else". Any proposed rules need to ensure this ideal is achievable, both "late type privacy checking + early checking of associated types" and "pure late type privacy checking + additional externalization of associated types" are good in this sense.

1 Like

One more interesting example involving both impl Trait and associated types.

#![feature(conservative_impl_trait)]

mod m {
    pub trait PubTr {
        type A: Clone;
        fn method(&self) -> Self::A;
    }
    
    struct PrivTy;
    #[derive(Clone)]
    struct VeryPrivTy;
    
    impl PubTr for PrivTy {
        // Indirectly leak VeryPrivTy in anonymized form.
        // Early checking doesn't help, we don't (and shouldn't) check impls for private types.
        type A = VeryPrivTy;
        fn method(&self) -> Self::A {
            VeryPrivTy
        }
    }
    
    // Leak PrivTy in anonymized form
    pub fn anonymize_priv() -> impl PubTr {
        PrivTy
    }
}

fn main() {
    use m::PubTr;
    // We obtained a value of PrivTy directly leaked in anonymized form.
    let anonymized_priv = m::anonymize_priv();
    // We obtained a value of VeryPrivTy indirectly leaked in anonymized form through associated type.
    let anonymized_very_priv = anonymized_priv.method();
    // We created a new value of VeryPrivTy indirectly leaked in anonymized form through associated type.
    let anonymized_very_priv2 = anonymized_very_priv.clone();
}

@nikomatsakis, what do you think about this?

(It looks like ā€œadditional externalization of associated typesā€ will be required anyway to do (non-pessimized) linking correctly.)

It's an interesting example, yeah. I think it itself doesn't violate my expectations, in that it seems similar to returning a trait object, but it does further serve as evidence that I can't really precisely say what those expectations are. That is, if your PubTr had a method like fn create() -> Self::A, then clearly main can create instances of VeryPrivTy "out of thin air" in some sense (it did have to be returned a PrivTy but...).

I'm confused a bit by this @petrochenkov. Let me re-summarize what I think I understood, and you tell me if I've got it wrong:

  • It would be nice if we could use privacy to allow us to elide symbols from binaries/rlibs/metadata, confident that downstream crates will not need to know about them.
  • Unfortunately, if a type returns an impl Foo, then in fact it might be a private type masquerading under there, so in some cases private types can leak out (this does not apply to leaking via an object).
    • We can however identify such cases pretty easily, no? (Well, maybe it's a bit tricky, but plausible.)
  • Similar things would then apply if it is possible to synthesize a path to a private type in generic code (e.g., <T as Trait>::Foo where T winds up being instantiated to PubType but Foo is bound to a private type), so we would have to consider all types bound in an associated type as being potentially visible from the outside.

Everything is correct :slight_smile:

Privacy is already used for this, but not so well as possible for trait impls.

Iā€™ve just been bitten by these rules. I have a program which does something like this:

enum DebugPrint {
  Print,
  NoPrint,
}

mod mir {
  pub fn build_and_write(..., print_llir: ::DebugPrint) {
    ...
  }
}

with code similar to this, I get the following warning:

warning: private type `DebugPrint` in public interface (error E0446), #[warn(private_in_public)] on by default
    --> src/mir/mod.rs:1151:3
     |
1151 |   pub fn build_and_write(mut self, output: &str, print_llir: ::DebugPrint) {
     |   ^
     |
     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
     = note: for more information, see issue #34537 <https://github.com/rust-lang/rust/issues/34537>

I canā€™t do anything about this, except make DebugPrint pub, which I really donā€™t want to do. Until pub(restricted) is actually implemented, warning about it seems really unfortunate and mean; it sucks to see this warning, and know that I have to make my code worse in order to get rid of a warning.

If it warned when you had something reachable, that would be different, but this is just a stupid, unfixable warning.

@ubsan

enum DebugPrint {
  Print,
  NoPrint,
}

=>

mod detail {
    pub enum DebugPrint {
      Print,
      NoPrint,
    }
}

+ maybe some reexports for convenience.

This is the officialā„¢ fix and itā€™s described in the tracking issue.