I think there's a fair amount of value in letting a field be mutable but only within the defining crate. Are you suggesting these accept a path? If not, surely that's a limitation worth considering.
Granted, but pub(crate) is widely used and I haven't seen anyone say "this is too much to write". mut(self) / mut(mod) is basically the same length (slightly shorter, but it's not just a single keyword).
Part of my concern with having the restrictions integrated into the existing visibility modifier is that this means what a valid visibility is depends on the type of item it's being attached to. While this would likely be rejected in AST validation, that pushes the error farther in the compilation process. This concern also applies to the :vis macro matcher; the extended syntax would be accepted even if the macro author doesn't desire this behavior. @scottmcm voiced these concerns, both in this thread and here, which is slightly after this sort of syntax was first mentioned. I agree with his concerns for the reasons stated.
Accepting it as part of visibility would also increase the size of a number of structs, namely those that don't even accept the restriction. This should always be secondary in decision making, however. The parser/AST could presumably be adjusted to avoid this drawback.
Formatting could help here. But this certainly doesn't apply when only adding one restriction (impl and mut can never be on the same item). Worth considering? Sure. But I don't foresee a future where there are a slew of restrictions that all apply on a single item.