This would be my concern. It feels like a serious performance footgun: anyone who can't correctly pick between Cell and RefCell (where we can just say "try Cell, falling back to RefCell if that doesn't work) seems unlikely to correctly pick between MagicCell and RefCell, which requires them to now also consider Clone performance.
What about using specialisation to have a RefCell-like type which, if it's used with a Copy type, also gains get, set, and loses the flag?