Proposal for promoting rustbuild to "official status"

[alexcrichton]

Good evening rustlers! Today I’d like to turn our attention to the topic of rust-lang/rust’s build system. We have historically been using makefiles for everything, but recently there have been efforts to rewrite it into a Rust/Cargo based world, dubbed rustbuild. This new build system is nearing completion and I’d like to see if we can push it the last mile to outright delete the makefiles! If you’re curious, you can find lots of motivation for this change on the original PR.

I believe that rustbuild is feature-complete enough to get in front of rustc developers today and I also think we’re ready to start moving over all official builds to it. To that end, I would propose the following schedule:

• Rust 1.15, starts 2016-11-10, releases 2017-02-02 - makefiles are deprecated
• Rust 1.16, starts 2016-12-22, releases 2017-03-16 - makefiles are deprecated
• Rust 1.17, starts 2017-02-02, released 2017-04-27 - makefiles are removed

In other words, the current nightly, 1.15, and the next, 1.16, we’ll message out deprecation of the makefiles. During this time we’ll switch infrastructure away from the makefiles (except for one bot to keep testing them), flesh out any final bugs in rustbuild, and give users ample time to switch. Once the 1.16 beta release is cut (2016-01-02), the master branch would become 1.17 and we’d delete the makefiles for good. This should give stable users around 6 months to ensure that rustbuild works for them.

So with that in mind, I’m curious for other’s opinions! Do you think this is an acceptable time frame for this transition? Do you think that rustbuild has critical bugs that still need fixing? As a reminder all bugs against rustbuild can be filed against rust-lang/rust and you can cc me (@alexcrichton) to tag it with A-rustbuild.

[brson]

Looks good to me.

[steveklabnik]

Rust 1.17, starts 2017-01-02, released 2017-04-27 - makefiles are removed

New year, no makefiles

I don’t do a lot of building the compiler these days but rustbuild has worked great for me.

[sfackler]

Death to makefiles!

[jer]

So far I have solely worked with the rustbuild system, so I’m totally in favor of getting rid of the old Makefiles.

For reference: There are currently 22 open issues tagged A-rustbuild.

[petrochenkov]

Do you think that rustbuild has critical bugs that still need fixing?

Yes, rustbuild needs ability to manually mark targets as completed (The rustbuild feature thread).

[matklad]

It might be a good time to write about rustbuild in the Contributing.md. I don’t use Rustbuild just because contributing.md talks only about makefiles.

[alexcrichton]

Yes, rustbuild needs ability to manually mark targets as completed (The rustbuild feature thread

Thanks for the link! Could you elaborate on the workflow you’ve got in mind here? I’ve often considered that basically moving inputs back in time is sufficient for messing with the build system (e.g. touch -t 199009090909 src/libstd/lib.rs).

Do you frequently need, though, to flag a step as being done as if you filled it in manually? In some sense this will be a very difficult problem to solve as almost all “real” steps that you’d want to skip are Cargo-based, and Cargo does not have this feature.

It might be a good time to write about rustbuild in the Contributing.md

Certainly! If we decide to move forward with this I’ll be sure to make a commit that:

• Updates ./configure to print out a big warning message if --enable-rustbuild isn’t passed
• Updates CONTRIBUTING.md
• Updates other docs I can find (e.g. the README)

[eddyb]

(e.g. touch -t 199009090909 src/libstd/lib.rs).

I prefer touch -d 'last year' because it’s far more convenient to me than anything using numbers, be them based on human calendars or UNIX timestamps.

[petrochenkov]

Could you elaborate on the workflow you’ve got in mind here? I’ve often considered that basically moving inputs back in time is sufficient for messing with the build system (e.g. touch -t 199009090909 src/libstd/lib.rs).

Suppose I have one run-pass test that fails on my machine due to bugs. Due to this failure I can never complete make check with rustbuild. What do I need to touch to effectively send all the run-pass tests “back in time” allowing make check to complete?

Alternatives like maintaining a stash with // ignore-test are very frustrating.

[eddyb]

I’ve suggested per-file “success” files that compiletest checks before (we sort of have that already? but they’re not checked to avoid running the test), although I can’t find an issue about it.

Not only do they solve rustbuild’s “running the tests every time even if they passed before” problem, but also “succeeded with a filter means tests doesn’t run again” (from the Makefiles).

[alexcrichton]

Note that rustbuild doesn’t cache test success right now the way the makefiles do. That is, ./x.py test will always run the full test suite unconditionally. Adding a feature like this to compiletest though sounds great!

[jntrnr]

TBH I was never a fan of the test file caching as half the time it’s not actually what you want and you have to touch a file to get the tests to re-run after you make the fix.

[eddyb]

But that’s only when it’s per-test-suite, not per-test-file, right? The latter should re-run by itself after making a fix.

[jntrnr]

I don’t remember the exact steps, only that I had to use touch a lot when I was working with the unit tests. It’s possible there was a better way, but I definitely prefer having thing less “smart”

[eddyb]

The problem is not that it wasn’t actually smart. Per-test success tracking would be more or less infallible - if we had it.

[alexcrichton]

Ok, I’ve opened up a PR to switch the defaults, so we can move discussion over there now.

@petrochenkov I figure that because rustbuild doesn’t even have the feature of caching test successes, then there’s not really a place to slot in a feature like you mentioned into rustbuild yet.

[petrochenkov]

@alexcrichton I’ll try to migrate to rustbuild once more and report how it goes. The last time I tried the aforementioned problem made it pretty much unusable, but maybe I’ll find some workaround this time.

[cuviper]

• Rust 1.15, starts 2016-11-10, releases 2017-01-02 - makefiles are deprecated
• Rust 1.16, starts 2016-12-22, releases 2017-03-16 - makefiles are deprecated
• Rust 1.17, starts 2017-01-02, released 2017-04-27 - makefiles are removed

I believe 2017-01-02 is supposed to be 2017-02-02 to make 6-week cycles. (Nov 10, Dec 22, Feb 2, Mar 16, Apr 27)

[alexcrichton]

Wow apparently I can’t count… thanks!