Representation of enums

[nikomatsakis]

Discussion topic about how enums are represented in Rust. Some things to work out:

• What are the #[repr] options available for enums?
  • RFC 2195 defined the layout of #[repr(C)] enums with payloads.
  • RFC 2363 offers a proposal to permit specifying discriminations
• When do we currently perform Option<T>-like layout optimizations?
• When do we guarantee Option<T>-like layout optimizations?
  • For any Option-like enum? What about things like Result<T, ()>?
  • What are the conditions on T? (Obviously, must have some undefined values)
  • the Rustonomicon has some text on this topic
• Are there important enum variant optimizations we want freedom to do in the future that we might want to keep in mind?
• Size of empty enums and !: defined to be 0
• C-like enums: define, what does it say about representation?
  • document the effect of #[repr(C) and friends here

[nikomatsakis]

I'm surprised to see so little activity here. I'm going to stake out a position that:

• We need to guarantee Option<T> is optimized for some range of types T
  • I think we should guarantee that this is true for any enum where you have one nullary variant and one non-nullary variant
  • I think the types where this applies should include &T, extern fn, and stdlib types like NonZeroU32 that are designed for this
• Otherwise, we should reserve the right to optimize (or not optimize) other sorts of enums as we choose

We also probably want to document (but perhaps not guarantee forever?) our layout rules around ! -- in particular, we currently say that if you have an enum with a variant whose types are "completely" uninhabited (e.g., excluding product types), then this variant is ignored. This is needed, iirc, to handle uninitialized data, where we may try to initialize and use just those parts that are inhabited.

[hanna-kruppe]

• I think we should guarantee that this is true for any enum where you have one nullary variant and one non-nullary variant

+1

• I think the types where this applies should include &T, extern fn, and stdlib types like NonZeroU32 that are designed for this

+1, follow-up question: should we document this as propagating through newtypes? (and what's the exact defintion of newtype for this purpose -- e.g., do PhantomData fields make a difference) late edit: perhaps we'd like to go even further and document it for any aggregate that directly contains one of these types?

I propose guaranteeing one additional fact about repr(Rust) enums: an univariant enum should be represented the same as a struct containing the same data as the sole variant. So for example:

• enum Foo { Foo } is laid out like struct Foo;
• enum Foo { Foo(i32) } is laid out like struct Foo(i32);
• enum Foo { Foo { x: i32, y: i32 } } is laid out like struct Foo { x: i32, y: i32 }

This would be particularly useful if we also guaranteed that fully-uninhabited variants are ignored for layout purposes, because it allows type-punning between T and Result<T, !>.

[gnzlbg]

What about Option<T> where T has enough unused continuous space to store the discriminant (e.g. in padding bytes) ? Could we / should we guarantee anything here ? We can always leave it unguaranteed for now. cc @eddyb

EDIT: Is there a general way to phrase which things allow us to do enum layout optimizations ? E.g. invalid representations, memory that's "invalid to use" (padding bytes), etc.

[hanna-kruppe]

As I wrote earlier in #13, I don't see any way to repurpose padding for these or other purposes (if it's settled to be padding in the contained type's padding -- while determining the layout of T we can futz with the padding between T's immediate members) because code that just writes to the T through a pointer wouldn't be obligated to preserve the discriminant/padding, nor would it even know how to do that.

[alercah]

I would like to see Result<&T, ()> get optimized, but this depends on the representation of (), and in particular, what if we consider platforms where null pointers are nonzero? Do we always make () have a representation such that it can be used in this way?

Newtypes with repr(transparent), at the least, should be able to be optimized in enums in the same way that their underlying types are. This is not something that we can fully commit to until we discuss validity of values, however.

I agree with @eddyb's distinction in rust-lang/rfcs#2363 that the tag and discriminant are different concepts, and only the RFC 2195 reprs necessarily make the discriminant directly affect layout. I think that repr(rust) should offer no guarantees about how the discriminant values are taken into account for layout.

In particular, if RFC 2363 is adopted, I am inclined to say that assigning discriminants does not affect the Option<&T> layout optimization.

[eddyb]

Explicit discriminants won't affect a two-variant enum, because only one invalid value is being used, but the choice of assignments does matter when more variants are involved.

In particular, if there any gaps in between the discriminants of variants that need invalid values, that will use up invalid values as well.
We have a pragmatic limitation, which is that decoding the discriminant from the invalid value representation should be simple/quick: currently it's limited to "rebasing" the invalid value to the discriminant (i.e. they have matching ranges, just different starts).

[alercah]

Can you elaborate a bit more on why this is the case? On a #[repr(Rust)] enum, is the discriminant used for anything other than std::mem::discriminant()?

[eddyb]

@alercah Yes, it's what the MIR switch (really called SwitchInt) works with, for matching on an enum. So we have to decode the tag/invalid value into the discriminant value.

[alercah]

Ah, I see. Why do we support specifying discriminants on repr(Rust) enums at all, then?

[eddyb]

@nox's usecase is matching the tags between two enums for optimization purposes (LLVM generating better code if they match up, without writing any unsafe code).

[alercah]

That usecase isn't repr(Rust), though. What they're trying to do wouldn't work in repr(Rust), since there's no guarantee that the discriminant is stored in the first byte (or, in fact, is stored anywhere, because niche optimizations may mean it doesn't have to be).

If I write enum Foo { Bar = 1 }, it's not clear to me what the = 1 is actually doing. I'd forgotten that std::mem::Discriminant doesn't actually expose the underlying value. So you mentioned that the compiler uses it internally, but as far as I can tell, since we don't offer any layout guarantees, there's no guarantee that Bar is actually represented as a 1 of any kind.

I think this means that we should just deprecate explicitly-specified discriminants on repr(Rust) enums.