I think it's a non-sequitur. But let me rephrase it this way: the fact that an operation is safe doesn't mean that it doesn't have unsafe pre/postconditions or doesn't rely on external unchecked assumptions (e.g. the classic Vec::set_len, or the closure pinning debacle). With your approach, safe code should be unconditionally moved outside of unsafe { }, which actually inhibits properly documented reasoning.
Just as some safe operations would be better placed inside of unsafe { }, so should individual operations sometimes be merged in a single block.