Compile rustc entirely from sources, no binaries

Hello!

I want to compile rustc completely from sources, with no binary blobs.

This is partly to satisfy my curiosity. It's also because I am concerned about the long-term implications of relying so much on binary blobs for rustc development. (For example, I have not been able to deduce the complete security story of blobs pulled by rustup.)

Is there a known happy path to compile rustc entirely from sources?

My best understanding is that I'll need to iteratively bootstrap versions of rustc until I get a working stage0 compiler. I'll need to start with a C compiler to bootstrap that first version of rustc.

If nobody else has done this, I'm happy to take advice and run experiments.

Thanks!

The old versions of Rust were written in OCaml, not C. You’ll have to decide if that’s good enough or if you have to bootstrap OCaml first...

3 Likes

There should be several stories around about how to bootstrap current Rust from a given compiler like for example GCC/g++.

Here's one description from 2018: https://guix.gnu.org/blog/2018/bootstrapping-rust/ and I hope it is about the same thing you want to do, building rust "from scratch". Building from OCaml would be a gigantic task, and the more recent ways like the one described here have actually been done.

I've heard rumors that mrustc support for compiling Rust 1.29 is in progress :wink: @thepowersgang

2 Likes

Yeah, back in the early bootstrapped days you have to almost compile each merge commit with the previous one, there was no snapshoting like now ("every stable Rust build with the previous stable Rust") as far as I know. So that'd likely be hundreds of bootstrap builds, re-tracing all of Rust's history...

mrustc is the much more promising approach.

I agree, AFAIK several people have made it work other than guix, including:

https://android.googlesource.com/platform/prebuilts/rust/+/refs/heads/master/bootstrap/mrustc-bootstrap/bootstrap.bash

Those rumours are true, and progress is good.

20 Likes

Is there something particular about rust that causes you concern, here? Do you trust the binaries that build your C compiler more for some reason? Is there something that C does better here that you'd like Rust to do?

(Not to mention the binary in your processor, or other such things...)

7 Likes

Sounds like mrustc is the way to go! I'll look into contributing to that project. Thanks everyone!

@scottmcm I'm not saying trust no binaries, I'm saying reduce the number that we have to trust. I'm aware of, and sympathetic to, security issues like the trusting trust attack. The fact that there are binary blobs in our CPUs doesn't imply that we want to use more binary blobs than we need to. (I believe this to be especially true in Rust's case, given the aforementioned questions around the security of rustup blob creation.) :-]

What I would find useful is if some small number of reputable companies would do an mrustc-based bootstrap from rustc 1.19.0 and, shortly after each official compiler release, post some small attestation that the official checksum matches the one that they get from their bootstrap chain. I think if Google, Facebook, Microsoft, and maybe one other reputable company were doing this as a community service, it would basically eliminate any concerns around simply using the official blobs even in use cases where that would ordinarily be far out of the question.

11 Likes

Is rustc reproducibility good enough to do this? It may be good enough now, but I think it certainly wasn't good enough in 1.19.0.

You don't need every old build to match. From what I've heard, the bootstrap chain converges to the official checksums after some point.

2 Likes

This sounds super-useful to do (checksum publishing etc) and would perhaps be something the https://reproducible-builds.org/ team might be interested in helping with?

This is not correct. We had a snapshot-registration system in place from the very beginning of bootstrapping. There have just been many, many snapshots over the years. Replaying even the snapshots would take a long time.

5 Likes

Thanks for chiming in and correcting what I misremembered. :slight_smile: