Enabling MIR by default

[brson]

I think crater is probably also not a sufficient bar for testing ‘correctness’ too. We should probably do a full run off all tests on crates.io on all three tier 1 platforms, especially windows.

[eddyb]

Can there not be a -Z launchpad for anyone that needs latest nightly and is afraid about such regressions?

There is, in the PR I mention both -Z orbit=off and #[rustc_no_mir] (the latter was used in tests to allow us to bootstrap with -Z orbit before everything was implemented).

[aturon]

Quite simply, Is not making it the default the best way to discover and fix (compiletime and runtime) performance regressions?

Eventually yes, but there’s no reason to take this step in a rush, when we can first work to mine data across a large corpus of Rust code like crates.io. Passing that bar first reduces potential pain/churn in the nightly ecosystem, and has already been serving well to spotlight issues with the transition.

[Ericson2314]

Fair. Performance wise, I assume Crater mainly gives information compile-time performance, and perhaps some run-time performance information from benchmarks and test?

[nikomatsakis]

I think crater is probably also not a sufficient bar for testing ‘correctness’ too. We should probably do a full run off all tests on crates.io on all three tier 1 platforms, especially windows.

I will try and get something running for correctness here. I agree it’d be good to cover windows better.

[nikomatsakis]

crater gives no information about timing of any kind.

[nikomatsakis]

I’ve been wondering how best to gather timing numbers. I’ve adapted a script from @jntrnr that can run and scrape timing information across all of crates.io, but I feel a bit nervous about building arbitrary crap from crates.io on my computer – seems like a security risk to do so, given that it may involve executing arbitrary code.

I can use an EC2 instance, but I’m not sure whether the numbers that would result are reliable in any way. Seems unlikely. The same might apply to a VM – though I’d expect the figures from a VM to be relatively ok, just a higher margin of error.

Perhaps a new user on my laptop would be a good compromise.

Thoughts?

[anp]

You could run the tool while booted from a live usb environment?

[brson]

You could run the tool while booted from a live usb environment?

This seems like a good bet, and will work for at least linux, you would want to do it while you are not connected to the Mozilla network. For macs I can sacrifice one of the decommissioned bots, but we’d want to be careful about its network connection - alternately we may be able to temporarily re-image a macstadium machine (which is off the mozilla network), or commission a new one of those. Not sure if Windows has live images for this.

[arielb1]

This seems like a good bet, and will work for at least linux, you would want to do it while you are not connected to the Mozilla network. For macs I can sacrifice one of the decommissioned bots, but we’d want to be careful about its network connection - alternately we may be able to temporarily re-image a macstadium machine (which is off the mozilla network), or commission a new one of those. Not sure if Windows has live images for this.

I prefer to use a VM manager. You could install Qubes (qubes-os.org) for Windows/Linux - a dedicated Xen provides fairly reliable timings.

[brson]

I prefer to use a VM manager. You could install Qubes (qubes-os.org) for Windows/Linux - a dedicated Xen provides fairly reliable timings.

Yeah, of course. VM is secure enough.

[sgrif]

Not the most scientific benchmark, but Diesel’s test suite seems to be compiling at around the same speed with MIR. 77.01s baseline, 76.81s with -Z orbit. Averaged over 3 runs, difference between fastest and slowest was ~1s in both cases. Anyway enough to say there appear to be no regressions there. (Full command was cargo clean && time cargo rustc --no-default-features --features="sqlite unstable" -- -Z orbit) Nice work!

[tomaka]

Compiling my personal project. Without MIR, 9m2.537s. With MIR, 7m52.177s.

[seanmonstar]

Similar reports compiling hyper: no perceivable difference. Same number of seconds.

[nikomatsakis]

So a status update. Before leaving for vacation, I did manage to get an EC2 instance up and running and to collect data running tests across all of crates.io – but I haven’t manage to collage that data yet. In case anyone wants to experiment, my test runner is adapted from @jntrnr’s repo which in turn is a bastardized version of cargo apply (by @alexcrichton), I think:

nikomatsakis/cratesio_perf

It’s not well documented yet but you can do things like run tests/benches across selected packages. Unfortunately the benchmark results aren’t really usable yet as the timings are not accurately reported.

[nikomatsakis]

Before leaving for vacation, I did manage to get an EC2 instance up and running and to collect data running tests across all of crates.io

To clarify, I do plan to put in some time to collate those results over next week or two – but I haven’t quite scheduled it in yet.

[dns2utf8]

I have a silly question: How do you enable MIR for just one project?

I wrote a .cargo/config file with the following content:

[build]
rustflags = ["-Zorbit"]

Is there a more direct flag or option in the Cargo.toml?

Regards

[kennytm]

You could put the .cargo/config in your project’s directory.

project/
    Cargo.toml
    .cargo/
        config

[dns2utf8]

I did do that, I was not sure if it was “the way to go” or if there were a option directly in Cargo.

As I keep thinking about it: Since the long term plan is to switch so MIR completely, it may not be such a great idea to introduce a new option to Cargo.

[nikomatsakis]

Just a note that I added this milestone to track blocking issues:

rust-lang/rust