There's a cost to doing so though, even independent of the developer cost of manually doing subcrate dependency tracking (and associated risk of using functionality without declaring a dependency). Specifically, if I later determine that a feature is now necessary, enabling it requires a clean rebuild of that dependency (and anything downstream of it); nonlocal dependencies are compiled without incremental since they're assumed to change infrequently, and while feature gates SHOULD be API additive, there's no such restriction on unexposed impl details.
For those reasons, it's generally recommended to only feature gate meaningful coarse chunks of a library crate; typically entire modules' worth of functionality and/or interop support with an upstream crate. The compilation/pipelining cost of smaller chunks of functionality typically isn't worth the cost of feature gating it.
IF the compiler did such tracking automatically, though, it could potentially be different.
This would generally fall under the banner of "MIR only RLIBs," IIUC. It's not that simple, though, because LLVM is (a significant part of) the slow part. Separate compilation of crates is "embarrassingly parallel," and slamming everything into one translation unit and then splitting it back into codegen units makes that parallelism harder to recover. Plus, now you need to track incremental for what you've already compiled from upstream if you want to avoid redoing that work, whereas with fully separate compilation it's just ambiently known what is available from the upstream libs (anything nongeneric and noninline plus maybe shared generics (opt-in, unstable)).
And especially on Windows/MSVC, codegen isn't necessarily the only slow part; for binaries with a lot of crates/CGUs, linking can also take a significant amount of time, and is typically redone from scratch for every build.
The high-level reality is that architectural improvements to benefit clean compiles typically hurt incremental and vice versa. While clean builds are still important, what really matters the most is low-to-medium optimization incremental builds, since that's what bottlenecks development iteration time. It's perhaps a bit reductive, but if you're only making a clean fullopt fatLTO build twice a month for distribution, it doesn't matter much if it takes more compute time to produce it.