I did like the idea (which might have been yours) that inference is already essentially allowed in irrefutable places, so that might be one way to make progress on this -- but that wouldn't apply for enums at all, so probably doesn't help OP.
Come to think of it, we could at least improve the diagnostic here if people try it: Suggest the correct name if `_` is used in a struct literal · Issue #98282 · rust-lang/rust · GitHub