One version of this that's been discussed before is "move-only" fields that would have this constaint.
Because the main constraint on doing layout optimization is reading through &s and particularly writing through &muts, that would give repr(rust) far more freedom to bit-pack (even arithmetic pack, if it wanted) enums together, unify discriminants across fields, misalign fields if helpful, etc, since it could run extra code in both cases.
Move-destructuring such types would also be fine even if the fields were non-Copy.
Flatness of a type seems like it would be a mess for generic code, since I can take references to a T that I put in a wrapper of some sort today. Making that work seems like it'd probably need another opt-out trait (like ?Sized), but historically we've strongly not wanted to have more of those.