I work in an embedded domain, and use a Rust Enum to represent the Fault State of the program, where this enum is placed at a special address in memory. I do this as there are a defined set of valid states, and allowing arbitrary states would be unsound (and can cause UB at the system domain level). However, a valid state is the state representing "No-Fault", which has the bit pattern 0.
I wish to represent this state as the None Variant for an Option, and leave no variant on the discriminant 0. However, the current guidelines allows None to chose any invalid representation, rather than requiring zero. This leaves me the interesting position of forcing the only valid niche to be zero (causing the same soundness issue), or represent the "No Fault" state, as a Fault itself, which would be counterintuitive.
This would seem to be a potentially common pattern, internally representing an empty state using a zero bit pattern
This is an immediate example which comes to mind for requiring Niche Variant Optimization to prefer the Zero state, rather than pick any one.
This would apply if and only the zero bit pattern is an invalid state for the type, and would choose that pattern for the optimization. If the zero bit pattern is valid, I don't see a reason to prioritize any further niches, as few others would have the same applicability (the zero bit pattern is special in more cases).
Enum layout optimizations are deliberately underconstrained to allow future development. I would be quite worried about painting ourselves into a corner by giving guarantees like this. For example, when the niche of a type covers bit patterns 0xffffff00..0x000000ff (wrapping around), it would be rather silly to choose the null value for None as that would make the remaining niche non-contiguous.
Also, see this document for what currently is and is not guaranteed about enum layout -- honestly it sounds like you are already relying on things we do not guarantee.
When you need very specific representations like you do, the default repr(Rust) is not for you -- that's where the more precisely defined (and less implicitly optimized) representations become important. Maybe there is room for some kind of mechanism to explicitly request a variant to use a certain niche?
I am using repr(C) on the enum type. (Specifically, repr(u16)). However, the problem is the selection of state in Option.
It seems to me that the Null state is most commonly a "special state", and more likely to represent the absense of something.
Maybe the preference should only apply to non-repr(Rust) enums, as the optimization to a Null State is usually only meaningful in FFI. Another Enum that could use this is the error type in kernel space. You wouldn't say return E_OK, because that is an oxymoron. Ideally, I'd want to say None for that as well.
Another possible alternative would be repr(NonZeroU*/NonZeroI*), which would forbid autoassignment of a zero-valued variant, and require that Niche-Variant Optimization selects this niche.
You should use Option<FaultWrapper> where FaultWrapper is a newtype of NonZeroU16, and provide conversions between Fault and FaultWrapper, or alternatively provide conversions between a newtyped u16 and Option<Fault>.
The guarantee as written seems problematic: what if the hardware used, for instance, 0xffff to represent "no fault"?
The enum type I meant is Option. It is (implicitly) repr(Rust), and thus generally not suited for when you want to get detailed guarantees for how things get represented.
Generally speaking, as mentioned, I mostly see the zero state being commonly the "Absence of state" . In the specific case, Zero is specified as the default state of the Fault Register, indicating "No Fault occurring, state reset". In the error example, zero is specified by Posix to be the "returned successfully" code (except in functions like open(2), which return something).
As for the point about Option, Option is already solidified with its use in FFI, particularly with pointer types. I'm simply proposing to extend its utility to a wider domain, by specifying the niche to use in the optimization which already occurs.
I'm still missing the motivation for customizing std's Option type instead of creating your own enum with precisely specified layout. In particular, this sounds like a perfect example of why we let you specify the discriminant values for fieldless enums (in addition to #[repr(C)] and so on). You could even access this enum exclusively through an API that always converts it to an Option for convenience; I'd imagine it needs a wrapper of some sort anyway to enforce the memory location.
Maybe some concrete code would help? What exactly is it you want to write where "custom enum you usually immediately convert to Option" isn't good enough?
The custom enum type would need the same exact form as the optimized Option herein proposed, which itself would require that I used a non-explicit repr(Rust), forfeiting the benefits of writing my own type. The layout of the Fault code is specified at a (much) lower level than rust code.
The reason I specific want to use Option is because the Fault Enum needs to represent the lack of a Fault state as a Fault itself, which is counterintuitive. Lack of a Fault is by definition not a Fault.
The memory location is constrained by a Magic Symbol and a linker script. You then copy the Fault out of an Atomic cell thing that does volatile reads.
We do already guarantee the layout of Option<T> for some types; specifically, scalar types annotated with #[rustc_layout_scalar_valid_range_start(1)] #[rustc_nonnull_optimization_guaranteed].
Obviously, those attributes are rustc internal and a part of privileged std, and not something we want to stabilize.
But could it make sense to extend #[rustc_nonnull_optimization_guaranteed] to work with not only scalar types, but also #[repr(Int)] enums that have a 0 niche?
If we wanted to make that attribute something usable by stable code, it'd probably look like #[repr(guarantee_option_niche(0))], with a check that the type is scalar (scalar primitive, repr(Int), or repr(transparent) of a scalar type) and has a niche at 0. Option-like enums (those we guarantee the nullable optimization for) would then be guaranteed to store their fieldless variant in the specified niche. Other niching operations would be free to use the entire niche.
That would be fine as well. I'm just looking for the option (pun intended). If I have to opt in for it, that would be fine as well.
I don't know how well this would work, but I did previously offer NonZeroI*/NonZeroU* repr annotations, which would force a zero-niche (and diagnose for enums that had a variant explicitly tagged with discriminant zero, when no explicit discriminants are tagged, the discriminant zero is never filled).
I wouldn't mind using an unstable annotation as the opt-in, I have to use intrinsics anyways (due to lack of std, and the low-level operations I necessarily perform). May be less useful out of my specific domain, though.