if true { 1 } else { 2 } &3 isn’t a block-like expression, so it isn’t even in the running. &3 cannot be anything but a pattern.
The problem you are describing is an ambiguity that one might see happening in expression statements. To deal with that Rust expects the LHS of a binary expression to not be block-like for it to be turned into a statement.
In any case, this poses no problems in match blocks.