Pre-RFC: User-provided default field values

[steffahn]

@atagunov

Update: the suggested scheme is a little awkward to support choice number (3) too. Suppose I write an all-pub struct such that I want to be able to add pub fields with defaults to in the future. So I want users to write S{a : 1, ..} . But I don't yet have any field with defaults. I'm then forced to add _marker : () = () . And make it pub . It works.. but is a little awkward.

Ah thanks for bringing this up!! This is a shortcoming I noticed, too, but forgot about while writing my last answer. A public _marker field is a no-go, so the case is basically not supported at all (unless, perhaps, if you start using doc(hidden) trickery to discourage usage of the _marker field).

[steffahn]

@atagunov

#[non_exhaustive] - means exactly that; today.

But the suggested scheme diminishes this ability.

If I write an all- pub struct with a single default I can no longer declare that.

You can still declare that, but differently: You’ll have to remember that – at least when adding the defaulted field (or perhaps better unconditionally to avoid any accidents in the future, hence my suggestion above to add a warning, if this is the course we take) - you replace non_exhaustive with the then more reliable alternative of having a private (dummy) field.

[jhpratt]

@atagunov

I'm then obliged to never add a private field, nor can I add a pub field without a default. unless I declare it a breaking change.

What alternative do you suggest?

I'm not entirely certain what you mean by the "three choices" linked. Your responses are far from clear to me. Code may be helpful to explain what you mean.

@steffahn

Except for structs without any = value defaults, (the exception is for backwards compatibility), #[non_exhaustive] plays the role as the …

Correct. In exhaustive structs where all fields are public and at least one field has a defaulted value, .. is permitted. I don't see why one wouldn't want this to be the case.

@steffahn

There is no … at all, but this may become possible in the future.

Correct.

@steffahn

For structs without any = value , #[non_exhaustive] keeps playing the role as a way to allow future addition of any fields (private or public, defaulted or not).

Correct. So long as there's no defaulted field, #[non_exhaustive]'s semantics remain unchanged.

@steffahn

I would however personally prefer introducing a new syntax (be it attribute or dedicated syntax) for this feature, to avoid the need for “stealing” repurposing the #[non_exhaustive] attribute.

Myself and the team member agreed that there should be a way to opt in/out for non-exhaustive structs, and I'm fairly certain this is something everyone can agree on. Quoting part of my discussion with the team member ("TM" here):

TM: I'm a fan of having clear ways of saying "this is what is and isn't part of my interface contract".

@jhpratt: But at the same time attributes galore isn't ideal.

TM: There's an argument, which I'm not sure how I feel about yet, that says we shouldn't make #[non_exhaustive] structs support .. without a further opt-in, because if we do then people end up committed to an additional interface contract unless they opt out with some new mechanism.

@jhpratt: And doing that would require users to add a private field, which is precisely why we introduced #[non_exhaustive] in the first place.

TM: There's also an argument, which I find appealing but risky, that we should support .. on #[non_exhaustive] structs, because in practice that's what people will typically want.

TM: And long-term we don't want people to have to list two attributes when in practice they only want one.

TM: An intermediate possibility would be to add something like #[non_exhaustive(..)] or however we spell an opt-in, but make that the default in a new edition and make you opt out in that future edition if you don't want it.

Take from that what you will. An attribute would almost certainly work, but my concern is adding more and more attributes to be able to change the semantics in a slight manner.

@steffahn

Regarding private fields, to name a theoretical use-case, note that (whether or not that’s desired), if private fields are supported, all structs with a const fn new() constructor could - technically - also start supporting initialization via StructName{..} . So we could make the standard library support things like

Setting aside how I actually feel about the ability to do that, I was more seeking a situation where it makes writing code objectively easier. I get what you're saying, though.

[atagunov]

Update: actually it's all about how to spell the opt-in.
I wanted to say that this

#[non_exhaustive]
struct S1{ a : u8 = 1 }

didn't look to me like a very good way to spell it. @steffahn has a better argument than me just bellow my post; his argument is that if we want to be future-compatible with covering private fields then the opt-in has to be spelled differently; and I agree with that.

what I meant by three options

Update: upon re-reading I have realised that option 1.1 isn't that useful for anything, maybe just allows .. syntax within the crate. So I'm not holding on to my argument any more. But Steffahn's argument still brings me to the same old conclusion: opt-in would better be spelled differently.

@jhpratt

I'm not entirely certain what you mean by the "three choices" linked Code may be helpful to explain what you mean

(1) currently this

#[non_exhaustive]
struct S1 { a : u8 }

allows crate author to add new fields without this being a breaking change
because crate users can't write `S1{a : 1}` this is a valuable
this is expressive
it is desirable that the meaning of the above does not change after your RFC

.

(1.1) I propose that this

#[non_exhaustive]
struct S1 { a : u8 = 1 }

mean exactly the same - that crate users still can't write `S1{a : 1}` nor `S1{a : 1, ..}` nor `S1{..}`
if they still can't it means that I, crate author, can make it ``` #[non_exhaustive] struct S1 { a : u8 = 1, b : u8 } ``` and it doesn't break user code

.

(2) this case is simple, these are `pub struct` without `non_exhaustive` with all `pub` fields
crate users can instantiate them as `S1{a : 1}`
it is a breaking change to add any fields
`S1{..}` syntax can probably be permitted

.

(3) this is the case for which you want to introduce an opt-in
I fully agree that there needs to be an opt-in
I'm happy to leave it to bike-shedding how to spell opt-in syntactically
Could be a new attribute
Could be `non_exhaustive(..)`
Could be no attribute at all - not even `non_exhaustive` but `..` inside struct def as @steffahn suggested:

struct S1 { a : u8 = 1, .. }

---

Bike-shedding starts: In fact I quite like @steffahn's suggestion - the last option above.

.

And I don't like very much the option of re-using `non_exhaustive` here. It seems more fitting to me to leave it for option (1) and especially option (1.1). It is the established meaning of `non_exhaustive`, maybe it's best to leave it as-is without change

---

I don't quite understand the resistance to S1{..} covering private fields with defaults. Especially if this has to opted-in to. But if it has to be a separate RFC in order for this one to pass it can be.

[steffahn]

One more point / clarification on my proposed syntax. Assuming that we (eventually) want to support private fields, as-well. (Or at least want to have a design where supporting private fields would fit in nicely.) And assuming that we do want, as I proposed, only support constructor calls on structs with private fields on an explicit, opt-in basis, then we would eventually need, as I layed out above syntax for both

@steffahn

• a way to opt-out of Foo { a, b } construction in a struct where all fields are public (i.e. as a way to avoid the pub(crate) _marker: () = () workaround)
• a way to opt-in to usage of Foo { a, .. } in a struct that has some private fields (in order to avoid any possible confusion/mistake if someone adds default values to those fields just with the intention of either using a custom value for the derive(Default) implementation, or in order to use .. internally)

While a #[non_exhaustive(foo_bar)]-style attribute can make sense for the first case, it doesn't make much sense for the second case. (As "non_exhaustive" suggests - in my mind, based on my knowledge what #[non_exhaustive] currently means - some kind of restriction for the user of the struct, not necessarily a way to opt in to new ways of constructing the struct being available to the user.) So in an attribute-based solution where the first point is spelled #[non_exhaustive(..)] and the second point is spelled e.g. #[constructible_with_defaults], when you have a struct

#[non_exhaustive(..)]
pub struct Bar {
    pub a: u32,
}

and now want to add a private field with default value (which is something you're allowed because of the "non_exhaustive(..)"), you would also need to change the attribute, so the struct becomes

#[constructible_with_defaults]
pub struct Bar {
    pub a: u32,
    b: u32 = 0,
}

Needing to change the attribute here seems, at least slightly, annoying. A unified syntax like

pub struct Bar {
    pub a: u32,
    ..
}

and

pub struct Bar {
    pub a: u32,
    b: u32 = 0,
    ..
}

has the advantage that you don't have to "change the attribute". The .. intuitively just tells you "use Bar { [field...], .. } to construct this struct!" In one case (all fields public) this statement it telling you "you cannot construct this struct (using the constructor) without using the trailing ..", in the other case (there are private fields, but all with default values) it's saying "you can construct this struct using the constructor and trailing ..".

Of course, the same advantage can be achieved if some new less "non_exhaustive"-saying attribute is used. Or perhaps you do feel like it's natural that

#[non_exhaustive(..)]
pub struct Bar {
    pub a: u32,
    b: u32 = 0,
}

would be supposed to mean "you can construct this struct "non-exhaustively" using the constructor with a trailing ... Also a (slight) advantage of dedicated ".." syntax is precisely that, that you - technically - don't introduces yet-another-attribute.

[jhpratt]

I had largely missed that response, to be perfectly honest. The conversion with the lang team member was last night; I just didn't bother to write up my comment until today so comments (including yours) made in the meantime were not considered in our discussion.

Personally I'm on requiring .. to be present in the struct definition (for exhaustive and non-exhaustive structs). It's clear and unambiguous, and as you mentioned allows setting a value for #[derive(Default)] without necessarily permitting Foo { .. } construction.

[josh]

For clarity, I was the lang team member in question. I happened to be talking to @jhpratt in a PM, but none of the conversation about this feature was private. I'm not speaking for the rest of the lang team, just my own position.

I do think that it's reasonable to want both "non-exhaustive but usable with .." (for if you might add fields but only public ones with defaults) and "non-exhaustive and not usable with .." (for if you might want to add private fields). On balance, I think the former is likely to be more common, and should have the briefer syntax (such as just #[non_exhaustive]). The latter (non-exhaustive with only public fields that all have defaults, but you don't want to allow ..) seems much less common, so it seems reasonable for it to have less convenient syntax.

I do personally think that we should not support .. for structs that have fields not visible at the point of the .. (so, private fields). In any case, I don't think we should tie that together with the RFC for ..; it's easier to process multiple independent RFCs than to process one RFC with several different things that don't absolutely have to be tied together. (I do agree that we have to handle .. and = value defaults at the same time, since they'd use the same syntax and it'd be a challenge to compatibly extend the use of that syntax later.)

[jhpratt]

@josh

For clarity, I was the lang team member in question

Confirming this.

@josh

for if you might add fields but only public ones with defaults

…

for if you might want to add private fields

A third, very real possibility is wanting to add a public, non-defaulted field. As this feature is new, it's difficult to predict whether authors will generally be adding fields with or without defaults to #[non_exhaustive] structs (though I do agree that adding a public field is more likely than a private one). Given the difficulty in predicting the future, I believe it makes most sense to require .. on #[non_exhaustive] structs in the same manner as on exhaustive structs. It's only a few characters, so it's not like it's a particularly onerous requirement.

[steffahn]

@jhpratt

Given the difficulty in predicting the future, I believe it makes most sense to require .. on #[non_exhaustive] structs in the same manner as on exhaustive structs.

For the record, even though my proposal above implies that an "additional" .. would be required on a #[non_exhaustive] struct in order to support initialization via a constructor with trailing .., it also implies that in this case the #[non_exhaustive] itself becomes irrelevant, and you need just the ... (To the point that we should definitely warn, perhaps even error, on that combination, suggesting to remove the then-redundant #[non_exhaustive] attribute on the struct). There's never the need to put multiple markers (i.e. both the #[non_exhaustive] and a ..) on the struct! This is also in line with

@josh

On balance, I think the former is likely to be more common, and should have the briefer syntax

[atagunov]

@josh

I think the former is likely to be more common, and should have the briefer syntax (such as just #[non_exhaustive]

...but that would change the meaning of existing code
ppl. have been using #[non_exhaustive] to mean exactly

@josh

"non-exhaustive and not usable with .. " (for if you might want to add private fields)

so far; perhaps @steffahn's syntax is indeed better - while also being more brief

[ekuber]

One thing that I didn't see (but might have just missed): If the following is supported

struct Foo {
    bar: Bar,
    ..
}

Then the following could also replace #[non_exhaustive] with a language construct:

enum Foo {
    Bar,
    ..
}

[steffahn]

@ekuber

Then the following could also replace #[non_exhaustive] with a language construct:

enum Foo {
    Bar,
    ..
}

If the syntax is supposed to mirror the use case (pattern matching) it would have to be

enum Foo {
    Bar,
    _
}

instead , in any case I guess that's mostly a discussion unrelated to this topic, right? (Also, one disadvantage of new syntax here is that it might (incorrently) suggest to the user that #[non_exhaustive] attributes are generally being replaced by new syntax, even though for structs, the two would coexist with different meaning.)

[scottmcm]

@ekuber

Then the following could also replace #[non_exhaustive] with a language construct:

enum Foo {
    Bar,
    ..
}

That's exactly what it was in Extensible enums by nikomatsakis · Pull Request #757 · rust-lang/rfcs · GitHub, BTW, which wasn't accepted. Then when it came back in https://github.com/rust-lang/rfcs/pull/2008, where it was accepted, it was an attribute instead.

I don't have strong feelings between .. in the definition vs #[non_exhaustive(..)], though.

One thing that might matter: I'd expect that Foo { a, .. } would work inside the defining crate with #[non_exhaustive] even without .. in the struct definition (if it has the necessary defaults).

[josh]

@atagunov

but that would change the meaning of existing code

It wouldn't change the meaning of existing code, because you would only be able to use .. on a structure that has = value defaults specified for each field, and you can't currently do that in existing code. So we could consider new code that adds = value defaults as opting in to the use of ...

[jhpratt]

@steffahn

implies that in this case the #[non_exhaustive] itself becomes irrelevant, and you need just the .. .

Good point. I don't think a hard error is necessary, but a warning would be reasonable.

@ekuber

Then the following could also replace #[non_exhaustive] with a language construct:

For enums, yes. Structs and variants, no, as it would require promising that all future fields are defaulted. Let's keep the discussion to default field values, though.

---

Is there general agreement that:

• .., if present, must be located after all fields are declared and must not have a trailing comma.
• A user must construct a value using .. if and only if .. is present in an external struct definition.
• A user may construct a value using .. if and only if .. is present in a local struct definition.
• A warning shall be emitted when .. is used in a #[non_exhaustive] struct. The warning should indicate that the #[non_exhaustive] attribute is unnecessary when .. is declared.
• .. provides a forward guarantee that all future fields shall be public and defaulted. Elision of non-visible fields may be supported in the future.
• .. may only be used to elide fields otherwise visible to the user.
• .. must be used when pattern matching against a type where .. is included in the definition.
• In field: T = value, the value shall be considered a const context that resolves to type T.
• Defaulted values may be used on fields with any visibility.
• #[derive(Default)] will use the defaulted field values.

I am considering structs and enum variants to be equivalent here, unless someone has a reason they should not be?

I believe that sufficiently summarizes the main points. If I'm missing something, please say so! If there is general agreement, I will update the RFC text to match what I've just described. A name for the .. syntax (for both definition and usage) would be nice to have too.

[steffahn]

Note that it's a totally valid use-case to have a struct with all public fields, no default values at all, and you still want to force users to construct the thing using syntax like Foo { field1: 42, field2: "bar", ..} (specifying all the fields and using .. nonetheless), i.e. disallow Foo { field1: 42, field2: "bar" }, because you want to keep the possibility to add new fields with default value in the future without it being a semver-breaking API change.

Hence I think this statement is wrong:

@josh

you would only be able to use .. on a structure that has = value defaults specified for each field

since for using .., only the fields that are not provided need default values, and if all (currently existing) fields are provided, then no fields need to have default values at all.

[Nemo157]

@jhpratt

Elision of private fields may be supported in the future.

Does this mean that private fields may not have defaults initially then? It seems like this would be a visibility change for the constructor of a struct with a mixture of public and private fields where they all have defaults.

[jhpratt]

@Nemo157

Does this mean that private fields may not have defaults initially then? It seems like this would be a visibility change for the constructor of a struct with a mixture of public and private fields where they all have defaults.

Changed to read non-visible fields. I specifically mention that

@jhpratt

Defaulted values may be used on fields with any visibility.

[Nemo157]

The case I'm thinking of is:

struct Foo {
  foo: i32 = 0,
  pub bar: i32 = 0,
  ..
}

With the initial definition that will be constructible only in-crate since it has a private field. If in the future the rules for .. are relaxed to allow eliding non-visible fields that would become constructible anywhere since all fields are defaulted. It feels like saying that that change may happen, without guaranteeing whether it will happen or not, makes cases like this very dangerous to use since you don't know what the behaviour will be in the future.

[steffahn]

@jhpratt

• .. , if present, must be located after all fields are declared and must not have a trailing comma.

Sure.

@jhpratt

• A user must construct a value using .. if and only if .. is present in an external struct definition.

Technically "constructing" can also be done using a function like fn new() -> Self, but assuming you mean "a user must add .. to a literal struct- or variant-expression if and only if ..", then that sounds about right.

@jhpratt

• A user may construct a value using .. if and only if .. is present in a local struct definition.

Okay, now that formulation confuses me.. looking ahead,

@jhpratt

.. provides a forward guarantee that all future fields shall be public and defaulted. Elision of non-visible fields may be supported in the future.

indicates that this summary is supposed to be about how things work if we don't allow usage of .. for private fields with default values. (Or more precisely for not-currently-visible fields without default values.)

So IMO this section

@jhpratt

• A user must construct a value using .. if and only if .. is present in an external struct definition.
• A user may construct a value using .. if and only if .. is present in a local struct definition.

Needs to be re-phrased. At least if this is supposed to follow the proposal I made, minus the part about private fields, then adding .. to a struct item works only as a way to disallow not using the .., never as a way to enable ... Well, unless perhaps for #[non_exhaustive] structs. (To be honest, IMO if we disallow .. on structs with private fields, then we should also disallow it on #[non_exhaustive] structs, i.e. the "it's a hard error instead of a warning" approach to struct items featuring both .. and #[non_exhaustive]).)

So following this reasoning, the rules would be

• A user can construct a value using .. if (and only if) all fields are visible and all left-out fields have default values in the corresponding struct/enum item, and the struct or enum-variant is not #[non_exhaustive], or local to the current crate.
• A user has to add .. to struct expressions or enum variant expression, if the structs/enums is external and declares .. in its struct/enum item (in the corresponding variant).
• A struct/enum item can declare .. (for a struct, or for an enum variant) if and only if the type and all of its fields are public, and the struct or enum-variant is not #[non_exhaustive].

By the way, I suppose we also only support braced structs or enum variants.

The proposed rules above of course imply that

@jhpratt

• A warning shall be emitted when .. is used in a #[non_exhaustive] struct. The warning should indicate that the #[non_exhaustive] attribute is unnecessary when .. is declared.

turns into an error (and also applies to enum variant). It does feel appropriate to force the user to remove their #[non_exhaustive] attribute if they want to give up their right to add private, or non-defaulted public fields to their struct in the future. Also if there is a need to allow this combination of .. + #[non_exhaustive] for whatever reason in the future, it's always backwards-compatible to remove this error.

---

@jhpratt

.. must be used when pattern matching against a type where .. is included in the definition.

Right, good point to mention that. The restrictions are the same as when .. has to be used for constructing the struct/enum, i.e. it also only applies to external structs/enums, i.e.

• A user has to use .. in patterns, if the structs/enums is external and declares .. in its struct/enum item (in the corresponding variant).

@jhpratt

• Defaulted values may be used on fields with any visibility.

Yes, indeed.

@jhpratt

• #[derive(Default)] will use the defaulted field values.