I ran another test, with a much smaller crate (commit hash c570a015e15214be46a7fd06ba08526622738e20), and got less consistent results, more in the noise.
Non-parallel rustc:
real 0m19.857s
user 2m33.522s
sys 0m9.018s
real 0m19.849s
user 2m30.636s
sys 0m8.092s
Parallel rustc (EDIT: ignore the first time measurement, see below for how this happened):
real 0m20.149s
user 3m30.696s
sys 0m15.814s
real 0m18.522s
user 2m42.483s
sys 0m10.206s
Because those two builds with the parallel rustc gave such wildly different results, I tried a few more builds with the parallel rustc:
real 0m18.837s
user 2m43.548s
sys 0m10.142s
real 0m18.509s
user 2m43.081s
sys 0m10.904s
real 0m18.613s
user 2m43.926s
sys 0m10.744s
Those three runs seem far more consistent, both in user time and in wall-clock time.
EDIT: oh, I just realized the problem, and it has nothing to do with rustc. The build of git2 (and potentially other libraries) compiled C code, and that compilation used ccache, missing the cache the first time and hitting the cache the remaining times. Nevermind, please ignore that first build. Leaving this up to document a potential pitfall for others doing build benchmarking.
Looks like smaller crates get a decent improvement in build time as well, though not quite as substantial.