Pre-RFC: Orthogonal Discriminants

language design
1

Summary

Right now the layout algorithm is mixed up with the type of the discriminant.

This RFC splits those up better for more flexibilty and new functionality.

Motivation

Quick! What's the difference between #[repr(Rust, u32)], #[repr(u32)], and #[repr(C, u32)]? What does #[repr(u128)] do in Rust 1.14? What does #[repr(u48)] do in Rust 1.138? Why is mem::Discriminant<Option<u8>> bigger than Option<u8>? How can I make a c_long discrinant? Is enum Foo { A = 2222222222 } something you're allowed to write?

The core problem here is that the repr attribute just isn't a great place for the type of the discriminant. Specifying it at all changes the layout algorithm. It's unclear what it's promising in semver. Proc macros that check layout algorithms need to know all possible tokens, so break for new types.

This RFC proposes splitting out the layout and usage choices

  • Are layout optimizations allowed?
  • How do fields in variants get laid out?
  • Can you cast values to integers?
  • Who can see the discriminant type you picked?

from the choice of which discriminant type it uses.

Guide-level explanation

It's possible to associate a number with each of the variants in an enum:

enum Symbols {
    Octothorp = 0x23,
    Ampersand = 0x26,
    Asterisk = 0x2A,
}

These are known as discriminants, and each must be unique.

By default they have type isize, so using a value of a different type will fail:

enum Symbols {
    Octothorp = b'#',
    Ampersand = b'&',
    Asterisk = b'*',
}

error[E0308]: mismatched types
 --> src/lib.rs:2:17
  |
2 |     Octothorp = b'#',
  |                 ^^^^ expected `isize`, found `u8`

You can, however, pick a different integer primitive for the type of the discriminants:

enum Symbols : u8 {
    Octothorp = b'#',
    Ampersand = b'&',
    Asterisk = b'*',
}

That type need not just be directly a primitive name; it can be a type alias or path. Notably, you can use things like : pub core::ffi::c_short in FFI scenarios. It does need to be resolvable to a specific primitive integer type at declaration time, however. It cannot be a non-primitive (like String or NonZeroU32) nor can it depend on any generic parameters of the enum.

If you're just matching the enum with match or if let, then the discriminants are irrelevant.

There's a couple of different ways you can make use of them, however.

From inside the same module, you can always use your_enum.#discriminant to get the value. The type of that value is whatever type you picked, or isize if nothing is specified.

The visibility on that is treated like a a field would be. Because it's private by default, you can pick whatever type and values are convenient without making a semver commitment.

This is particularly useful inside derive macros, as generated trait impls can thus get the discriminant values from instances without needing to replicate the layout algorithm.

For example, on a fieldless enum a derive could produce an implementation like

impl Ord for TheEnum {
    fn cmp(&self, other: &Self) -> Ordering {
        Ord::cmp(&{self.#discriminant}, &{other.#discriminant})
    }
}

that will compare using the declared discriminant type without needing to write it anywhere.

If you do want to promise something, however, you can specify a different visibility. For example, on a publicly-exported type you can say

pub enum Symbols : pub u8 {
    Octothorp = b'#',
    Ampersand = b'&',
    Asterisk = b'*',
}

which promises that the discriminant type will stay u8 and that the values of those discriminants will stay the same. That also lets downstream consumers use .#discriminant. (Other visibilities like pub(in some::path) are supported as usual too.)

On it's own, specifying a discriminant type doesn't impact layout. The compiler is allowed to look at the specified values and pick a smart encoding to save space.

There's a variety of repr options you can use for more control.

repr(fieldless)

A fieldless enum is similar to a transparent struct around the discriminant type: variants are allowed to have 1-ZST fields, but nothing else.

This guarantees that the size and alignment of the enum type match that of the discriminant type, and the allowed values are exactly those of the discriminants. (Note that this still doesn't allow using arbitrary values of the discriminant.)

Additionally, this allows as casting the enum to any primitive type from anywhere the discriminant type is visible (so long as the enum doesn't implement Drop).

repr(C, union_of_structs)

This uses what RFC#2195 called repr(Int); see that RFC for details.

repr(C, struct_with_union)

This uses what RFC#2195 called repr(C, Int); see that RFC for details.

Migration guide

You're using repr(Int)

If you have an enum like

#[repr(u32)]
pub enum MyEnum {
    A(…),
    B { … },
}

then you should migrate to

#[repr(C, union_of_structs)]
pub enum MyEnum: u32 {
    A(…),
    B { … },
}

(Or potentially to pub u32 if you wish to make a commitment.)

If you have an as-castable enum like

#[repr(u32)]
pub enum MyEnum {
    A = …,
    B = …,
}

then you should migrate to

#[repr(fieldless)]
pub enum MyEnum: pub u32 {
    A = …,
    B = …,
}

You're using repr(C, Int)

If you have an enum like

#[repr(C, u32)]
pub enum MyEnum {
    A(…),
    B { … },
}

then you should migrate to

#[repr(C, struct_with_union)]
pub enum MyEnum: u32 {
    A(…),
    B { … },
}

(Or potentially to pub u32 if you wish to make a commitment.)

If you have an as-castable enum like

#[repr(C, u32)]
pub enum MyEnum {
    A = …,
    B = …,
}

then you should migrate to

#[repr(fieldless)]
pub enum MyEnum: pub u32 {
    A = …,
    B = …,
}

You're using as casts but not a repr

If you have an as-casted enum like

pub enum MyEnum {
    A = …,
    B = …,
}

then you should migrate to

#[repr(fieldless)]
pub enum MyEnum: pub isize {
    A = …,
    B = …,
}

or consider picking a better discriminant type so lints can better guide your users.

Reference-level explanation

Syntax

The production for an Enumeration is currently

enum IDENTIFIER GenericParams? WhereClause? { EnumVariants? }

That changes to

enum IDENTIFIER GenericParams? EnumDiscriminant? WhereClause? { EnumVariants? }

with

EnumDiscriminant → : Visibility? TypePath

A new alternative is added for ExpressionWithoutBlockNoAttrs of

EnumDiscriminantExpression → Expression . #discriminant

Static Analysis

The path in the discriminant type must resolve to a primitive signed (i8, …, i128, isize) or unsigned (u8, …, u128, usize) primitive type at declaration time.

When resolving the type path in EnumDiscriminant, generics from the type are in-scope for name resolution, but it's an error to mention them.

The expressions in EnumVariantDiscriminant are type-checked with the expected type matching the previously-resolved primitive discriminant type. (The same as when it was spelled repr(Int).)

For expr.#discriminant:

  • the type of expr needs to known to be an enum. (It cannot be generic.)
  • the type of the result is the discriminant type of that enum.
  • the location needs to meet the visibility restriction.

For reprs:

  • If the EnumDiscriminant is included, no Int can be included in the repr.
  • If the EnumDiscriminant is included, C cannot be included in the repr unless there's exactly one of struct_with_union and union_of_structs.
  • struct_with_union and union_of_structs can only be used with C and with the EnumDiscriminant included.

Lowering

.#discriminant translates to the existing Rvalue::Discriminant in MIR.

Drawbacks

Changing discriminant types from tokens (which proc macros can rely on directly) to paths (which need name resolution) can introduce opportunities for confusion and hygiene complications.

For example, this will be legal with this RFC:

type u8 = u32;
enum Foo: u8 { A = 123456 }

Similar problems already exists in current rust, however, such as in struct fields.

Rationale and alternatives

Why require C for guaranteed layouts?

Today you can only pick your discriminant type by disabling layout optimizations. Safe Rust code, though, doesn't want to give those up but might still want to pick discriminant types and values.

Future work could also add more options, like repr(linear, union_of_structs).

Why add fieldless?

Today it's possible to as cast any enum that happens not to have fields.

That's not the way Rust usually does things, though, since it's something that you can commit to by accident. By having an explicit "yes, I want this" the compiler can check to make sure you don't accidentally add a field that disables it.

It's also helpful to give a direct migration path for types that are as cast today without needing to support that on other kinds of enums.

Why include visibility?

Without visibility, adding a mechanism for typed discriminant access opts-in everything to a new semver commitment with no opportunity to get ahead of it.

That's also why it's not just accessed via some trait method. (Of course, someone can offer a trait derive to expose it generically if they so wish.)

Notably, there's no way today to get discriminants as integers from repr(Rust) enums that have any fields in variants, and type authors should have a chance to consider what type and values they want to expose, if any.

For example, today Result has isize as its discriminant type. That's clearly not the type that a two-variant type would pick, but today it's impossible to change it to something else since that would remove the layout optimizations. It likely doesn't want to commit to any discriminant type for now, since bool would likely be the desired choice but this RFC doesn't allow that.

Similarly, it might not want to commit to Ok(()).#disciminant being zero either, as aligning discriminants with Option may allow more efficient conversions. (Result of course needs to preserve the stable behaviour of PartialOrd and Ord, but so long as there's no access to the discriminant order, they can change.)

Why include generics in name resolution but not allow using them?

We may in the future be able to allow them to be used, but adding them to name resolution later would be a breaking change so we can head that off now.

Given the different naming conventions, it's unlikely that this restriction will impact any non-contrived code.

Why the # in the pseudo-field?

Including a # emphasizes that it's something special, rather than a normal field ident.

That keeps it out of the way of any future features, like enum variant types, which might allow accessing variant fields directly.

Why put the discriminant type there?

As described in the prior art section below, this location is common in other braces-based languages.

A trait definition uses the order

trait Foo<Generic> : Base where Generic: Bounds { … }

so this uses the matching order

enum Foo<Generic> : DiscriminantType where Generic: Bounds { … }

and a super-trait is a TypePath just like the discriminant type is.

Prior art

Various other brace-based language use a similar syntax to pick the backing type of enums.

For example, C23 has

enum byte : unsigned char { … };

C++ has

enum class Handle : std::uint32_t { … };

C# has

enum ErrorCode : ushort { … }

Unresolved questions

  • Should expr.#discriminant be a value expression or a place expression? For now it'd probably have to block any non-copy uses of the place, but for known layouts that force it to be non-niched, we could potentially allow getting a pointer to that field specifically.

Future possibilities

More Types

Obvious candidates to enable would include things like bool, char, NonZero<Int>, and pattern types.

Generic Types

It could potentially be useful for discriminant types and values to be based on generic parameters.

That will introduce a bunch of complications, though, since for example it'll mean that the Discriminant rvalue in MIR is no longer simply offered

More special items

Like the foo.#discriminant expression, we could offer things like Foo::#DiscriminantType.

More layouts

If we add repr(linear) for structs, we could have repr(linear, union_of_structs) for enums.

We could also add more partial layout guarantees. Perhaps a repr(Rust, discriminant_firat) that still allow layout optimization with the constraint that the discriminant would exist at offset zero with the specified type.

19 Likes
2

Everything looks really good. I like the idea of this. I think, however, that the motivation section could use some concrete examples for why this is important.

3

Note that to be compatible with allowing #discriminant to be a place in the future, we need to prohibit &enum.#discriminant instead of making it borrow a temporary (this might be implied by "For now it'd probably have to block any non-copy uses of the place" but I wanted to be explicit).

1 Like
4

This looks great, and I look forward to having it, especially having a way to access the discriminant.

Nit: I think the names struct_with_union and union_of_structs are inconsistent, and I think we should consistently use of there: struct_of_union and union_of_structs. Otherwise, I think people will regularly forget which preposition goes with which.

Also, I think we talked about using .#tag or similar, rather than .#discriminant? That would be much shorter and easier to deal with.

1 Like
5

Will foo.#discriminant let me write to the discriminant as well? Will this only be possible on nicheless enums or in general on repr Rust enums too?

I can see uses for (unsafely) being able to write the discriminant and the data separately on a MaybeUninit<EnumType> in for example deserialisation. This is really not possible in general currently given all the complexity of niches etc.

6

What's the migration for "you're using repr(C)"? (in both the fieldless and fieldful cases) Is it "keep using repr(C) for fieldless (but as casting is deprecated), and add struct_of_union for fieldful"? I think that's fine, but it should definitely be included.

All the current Reference has to say is "the default enum size and alignment for the target platform's C ABI", but that is neither guaranteed by Rust to be core::ffi::c_int, nor guaranteed to be what the C compiler will do (a C enum's underlying type and thus its layout depends on its enumerator values, wildly).

3 Likes
7

This might want to at least discuss the interaction with niches.

The current repr(Rust) enum layout algorithm is able to lay out enums in such a way that all but one variant is represented by a concrete discriminant, but the remaining variant has an "everything else" discriminant value (and this layout saves memory if the "everything else" variant is strictly larger than all the others). This is implemented by placing the discriminant within the enum in such a way that it overlaps a field in the "everything else" variant that has a niche, and using only discriminant values that would be invalid in the niche.

I can see two potential problems with the interaction between niches and this proposal:

  • If an enum is making use of the "everything else" discriminant, its type needs to be the same size as the enum field that has a niche. This can make it difficult to specify explicitly, especially if the field in question has a type with a non-public representation.
  • It seems plausible that new enum layouts may be added in the future, that make use of niches in different ways. (Two examples include the "alignment niche" that exists in references to types that have an alignment above 1, and merging multiple enums that have disjoint discriminants into a single enum without adding a separate discriminant.) So it's hard for a proposal like this to interact with potential future layout optimizations.

The simplest way to resolve the interaction would be to, at least for the time being, forbid specifying the discriminant type of repr(Rust) enums, in order to allow for the possibility that the discriminant isn't represented as a field at all. Alternatively, you could define that specifying a discriminant type disables niche optimization, although I'm worried that this may cause people to accidentally define enums that have unnecessary discriminants.

One other problem is that this proposal may encourage people to mistakenly use discriminants that are too small, causing a performance loss. The pre-RFC, talking about Result, says "bool would likely be the desired choice", but it's actually a pretty bad choice for most Results. The reason is that it is usually undesirable for an enum to contain padding, because the compiler can't make assumptions about which values are in the padding bytes (meaning that, among other things, it nests less efficiently into other enums because it has fewer niches). It is almost always better to expand the discriminant to cover any remaining padding. (For example, an enum with variants None and Some(u8, u16) would want a bool or u8/i8 discriminant: it has alignment 2 and its largest variant has an odd size, so the discriminant should also have an odd size to add together to an even number. But Option<(u8, u16)> would want a u16/i16 discriminant, so that the even-sized discriminant can add together to the even-sized largest variant in order to fit into an alignment-2 enum without padding. These cases are different because the tuple (u8, u16) is four bytes wide rather than three.) This sort of phenomenon leads to counter-intuitive results like Option<(u8, u32)> being 12 bytes wide but Option<(u8, u32, bool)> fitting in 8. (You could perhaps fix this for repr(Rust) by requiring any padding bytes next to the discriminant to always be 0, rather than allowing them to be uninitialized; the only performance cost of this would be on zero-extending the discriminant when writing it, which is usually pretty cheap, but it might still be slower than using a sufficiently wide discriminant would be.)

2 Likes
8

I assume you meant discriminant_first

9

Does this mean that when migrating, users will need to add this repr to any fieldless enum? I have a concern that it may be too much. Would it be what bad to not require this particular migration, but allow it to use a default?

10

Not needed on all fiedless enums - only those where you use an as cast to convert them to their discriminant.

As long as you don't read the discriminant as an integer (e.g. you only use match), you don't need to change.

2 Likes
11

Rather than adding a new mechanism for enabling as casts, could this be an opportunity to deprecate them for enums? For discriminant-ascribed enums, given there is an explicit way of getting the discriminant, as casts could be unsupported, and perhaps over an edition we could migrate more/all enums to require this kind of discriminant ascription, given them a non-as mechanism for conversion?

3 Likes
12

Good question; I should add a motivation section about this.

I intentionally didn't do that because I want crates to be able to move to the new way without it being a semver break. If we deprecate as on the new way, that would prevent people from migrating without a major bump.

But yes, linting at the use site of "hey, consider .#discriminant instead" (or at least "don't as u8 a :pub u32 enum") would make sense.

13

That makes sense.

I still wonder if there's something interesting we could do at an edition though - I'm not familiar with the edition internals, but is it possible to expose as-casts only to certain editions, such that if both the defining crate and consuming crate are >= 2027 as-casts aren't available, but if either isn't they are? I assume this would be relatively novel, but it feels not wildly dissimilar to 2024's into_iter?

14

That still makes things a breaking change, because if I move my crate to 2027 and call something still on 2024, then that dependency upgrading will break me without me changing.

We can lint based on the edition of the crate with the as cast, but not really anything else.

15

Thanks! Overall this sounds great to me. However I was very surprised to learn in the middle of the RFC that this also proposes a new way of getting the discriminant of an enum; that should be mentioned in the summary and/or the motivation.

I assume this should use the same check as "fields that are trivial for repr(transparent), which is more restrictive than just "must be a 1-ZST".

This paragraph seems to imply that it is desirable for Result to specify a discriminant type. Why would that be? I think Result is doing fine as-is and see no reason to give it a (private) discriminant type even if this RFC gets accepted.

It would be good to explicitly say what happens with #[repr(C)] (without an explicit Int). Currently this results in an enum whose discriminant type on the Rust level is isize, but the actual layout is computed to match the "equivalent" C enum. See this PR for the details; the reference is currently incorrect here. I assume this isn't supposed to change since the RFC is all about the type observed via #discriminant, not about the actual layout.

Also, there's the elephant of mem::discriminant in the room next to #discriminant; a bit of a comparison would be good.

Note that "tag" and "discriminant" both exist as concepts around enums and they are distinct. Unless you propose to fully remove "discriminant" from our verbiage and use "tag" everywhere, I think it would be confusing to have two terms for the same thing (and we would be left with no term for what we currently call "tag").

Yeah the Reference is just wrong and my PR for fixing that will celebrate its first birthday soon.

IMO the RFC is fairly clear about the fact that there is no interaction with layout.

For context: The way the discriminant value actually gets stored in memory is (for non-repr(Int)) entirely unrelated to the type that you see after loading the discriminant. Even without niches, a repr(Rust) enum can have a u8 tag and an isize discriminant just fine (in fact that's exactly what happens today if you write enum Bool { False, True }).

So with this RFC, if you write enum Bool: u32 { False, True }, you get a u32 discriminant and a u8 tag. And enum Option<T>: u32 { None, Some(T) } will have the exact same layout as a normal core::option::Option, the only difference is that mem::discriminant will return the discriminant value (0 or 1) as a u32 rather than an isize, after decoding it from the niche. The actual value stored in the niche is already largely uncorrelated with the discriminant value you see, e.g. in Option<bool> the discriminant 0isize gets encoded as the byte sequence [0x02].

(Currently it is impossible to have a type with niches that has a non-isize discriminant so we'll have to carefully check those codepaths in rustc. But that's an implementation problem, nothing users should even notice.)

2 Likes
16

Do discriminants have better optimization than a plain method using match on the enum? Is the difference just that discriminants are checked by the compiler to be unique?

17

matching on the enum is compiled to checks against the discriminant.

Manually checking discriminants in situations where the you could just match directly is just making the code uglier for no reason; there's no reason to do that.

(The point of direct discriminant access is to avoid things like match x { A => 0, B => 1, C => 2, … } if you do want the integer for some reason. For example, derive(PartialEq) could choose to notice repr(fieldless) and just emit self.#discriminant == other.#discriminant, rather than a match against everything.)

2 Likes
18

Wow, I had assumed it’s compiled to checks against the tag… so I suppose tags get introduced at a lower level of abstraction than the part where matches are lowered?

19

Well, as always it depends on what level you're looking at.

match lowering happens in MIR where the code checks discriminants. That means, for example, that the discriminant for Option is always isize, even though it's extremely common that that's not the type actually used for the stored tag. It has to work that way, since MIR can be generic so it doesn't -- and can't -- know what the stored tag will be at that point. And then in codegen, this works by using the stored tag to calculate the value for the discriminant, and matching that value.

All that said, though, I've done to a bunch of work -- https://github.com/rust-lang/rust/pull/144764 and Simplify discriminant codegen for niche-encoded variants which don't wrap across an integer boundary by scottmcm · Pull Request #143784 · rust-lang/rust · GitHub for example -- to make sure that this extra dance usually optimizes out (even at opt-level=1) so the instructions you actually execute are just the obvious checks against the tag.

2 Likes
20

I am proposing that nobody wants to write out .#discriminant for a relatively common operation, and that outside the compiler people don't care about the difference between "the tag" and "the value of the tag / the way the tag is encoded in memory". We already use the two terms interchangeably in various places in both documentation and code, and people in the broader ecosystem often use them synonymously; only a subset of compiler internals distinguishes them.

1 Like