This is definitely a good idea, not just for security reasons.
Note that the continuous testing that we do for incremental compilation (https://travis-ci.org/rust-icci) already today relies on binary reproducibility of LLVM bitcode and object files. This has worked reliably on Windows, Linux, and macOS for a couple of months now.
I’m not sure about crate-metadata, but there have also been some changes that should make it pretty much deterministic. This needs to be verified though.
And then there’s the linking step. There’ve been reports that this might not be entirely deterministic on all platforms. But it could also just have been unstable crate metadata being misinterpreted as the linker’s fault.