The difference between aliasability and mutability is about closure captures, not about interior mutability.
Borrow checker knows nothing about interior mutability, IIRC, that’s why interior mutability primitives are implemented with raw pointers.
This comment describes why immutable but not aliasable data is used: https://github.com/rust-lang/rust/blob/6d620843f62b6cf3182528ffcaa877eba461bfbb/src/librustc/ty/mod.rs#L577-L612
As the comment explains, this is not strictly necessary, the language could require mut annotations from users instead, but author of the current scheme (something tells me it was @nikomatsakis) decided that this would be too cruel.