IMHO Cargo should distinguish between build products and build temporaries (e.g. Xcode has such distinction, and I suspect most other build systems). Currently both are dumped in the same place, making it impractical to point caches to a dedicated temp directory, without losing quick access to build products (i.e. crate's own executables and own top-level libraries).
There's an issue for this, although it suffered from a bikeshed about Linux standards:
Once Cargo is able to put temporary files in a proper dedicated system-level temporary directory, these files won't be causing headaches for backups any more.