This isn't a "probably shouldn't", this is a "must not". There are existing library interfaces that depend on the exclusive nature of &mut for correctness.
This isn't just an issue of missing optimizations. This is an issue of correctness; Rust with non-exclusive &mut would break existing code. We're not going to add an incompatible dialect of Rust that only works if you pass a non-standard compiler option, and that breaks the assumptions of existing Rust code.
You can have shared mutability, by labeling it appropriately, using an abstraction built atop UnsafeCell or atop raw pointers. That abstraction, by design, needs to have a different type, so that Rust code knows if it can rely on exclusive mutability. Code using &mut T may be relying on that exclusive behavior; code using MySharedMutableCell<T> isn't.