Optimizing by default

[tanadeau]

There has been some good discussion on (at least) two issues https://github.com/rust-lang/rfcs/issues/777 and https://github.com/rust-lang/cargo/issues/784 regarding whether or not to do optimized builds by default, and if not, how to let new users know that rustc my_file.rs or cargo build doesn’t give fast code.

I was curious for the community’s thoughts on this. I would like optimization by default with explicit flags (e.g., -O0 or --debug) to turn off optimizations. I think that would help to prevent “Rust is slow” first impressions that have also plagued other native languages including C++, Haskell, and Swift. Users coming from a Java, Python, etc. background may not even know that optimizations are a thing that they ever need to worry about.

If optimizing by default is controversial, I’d like to at least have rustc and Cargo print prominent warnings when no optimization flags are given.

As noted in the above issues, this has bitten newcomers with many issues on dev forums solved by "compile with -O".

[DanielKeep]

I like the idea of having Cargo print a single line message about which version it’s compiling. Preferably in a different colour to make sure new users notice it. For example:

> cargo build
 **Building for debug without optimisations.**
   Compiling thingy v0.0.0
[millions of warnings about unstable features]
> cargo build --release
 **Building for release with optimisations.**
   Compiling thingy v0.0.0
[oh goodness, more warnings]

[nstoddard]

One thing to keep in mind is that optimization is far more important in Rust than for most languages. In most languages, optimization provides a nice speed boost, but most programs will run just fine without them, just a bit slower. In Rust, optimization has a huge impact, often resulting in orders-of-magnitude performance benefits. If we decide not to do optimization by default, we’ll need to make sure it’s very clear to users how important optimizations are. Otherwise we’ll get tons of complaints and bug reports about performance problems simply because users didn’t enable optimizations.

I almost never run my Rust code without optimizations - it’s just too slow to be bearable, especially for things like games. Personally, I’m in favor of optimizing by default. As far as I can tell, the only downsides to optimization are somewhat slower compilation time, and difficulty debugging. The slower compilation time is more than offset by the faster speed at runtime, and debugging is a special case; I run my code normally much more often than I debug it.

[matthieum]

Interesting, I am in a completely opposite situation: I only ever run Debug when developing and only switch to Release when comes the time to validate the development (ie, just before submitting the PR).

In my write-compile-run loop, compile-time is therefore more often than not the single bottleneck…

Anyway, regarding how to advertise which profile was used to compile something, my proposal would be to simply put the generated binaries/libraries in a sub-directory bearing the name of the profile. When you launch:

user$ ./debug/mythingy

It tends to be somewhat obvious that it’s the debug version.

[Gankro]

One thing to keep in mind is that optimization is far more important in Rust than for most languages. In most languages, optimization provides a nice speed boost, but most programs will run just fine without them, just a bit slower. In Rust, optimization has a huge impact, often resulting in orders-of-magnitude performance benefits.

I do not believe this is fundamental. I’m reasonably confident that this is a consequence of the way the compiler is currently architected. The “unoptimized” code we generate for LLVM is quite naive.

[tanadeau]

Based on this and the discussions in the issues, I’m thinking I’m going to create issues in Cargo to:

1. Have a relatively “loud” notification of the current build profile when compiling as per @DanielKeep
2. To put the artifacts in a ./debug or a ./release directory in the target. Thanks @matthieum for the latter idea.
3. Have a warning or notice when running cargo bench without optimizations

There should also probably be a few additions in the Rust book when it mentions rustc (such as for the hello world example) that it is not optimized by default.

I’m hoping the combination of those will make accidentally trying to benchmark debug binaries a lot rarer.

[iopq]

cargo bench doesn’t do an optimized compile? What’s the point of benching then? I would have assumed the default is to do an optimized build and bench that

[Gankro]

cargo bench implies optimization (-O3 iirc) unless you actively override the defaults, so I don’t think that’s a serious concern.

[tanadeau]

@Gankro, yep, you’re right. That was fixed with https://github.com/rust-lang/cargo/pull/734. I was getting confused between cargo bench and the discussion at https://github.com/rust-lang/cargo/issues/784.

[kornel]

+1 for target/debug

I’m not sure about warnings. They get tiresome. Could you warn only once? Or only warn if Cargo.toml version is 0.0.1?

Perhaps cargo build should be replaced with cargo release and cargo debug?

[tanadeau]

They shouldn’t be warnings, just notices like the ones cargo currently gives like Compiling foo v0.0.1. They should be in a nice, pleasing color like green or blue

cargo release and cargo debug sound to me like they would release to crates.io and start a debugging session, respectively, instead of building.

[kornel]

Still, a line with such information, even if it had a pleasing color, loses its utility after a couple of runs and becomes noise.

You’re right that cargo release could be confused with cargo publish, etc. Maybe we can bikeshed better names?

[tanadeau]

My main concern is with new users coming from languages where optimization isn’t really user-visible. If the notice sticks out for them so they learn the difference, I think that alone would make it worthwhile even if it become “noise” later on. Even then, it would still be useful in build logs to check configurations.

When it comes to cargo subcommand changes, I would prefer there not being a default at all so users have to choose cargo build release or cargo build debug.

[tanadeau]

I’ve created issues in Rust and Cargo for doc fixes and logging of the build profile.

Issue: Intro of rustc in docs should mention debug/optimization

Issue: Note build profile in `cargo build` output

[Valloric]

I think optimizing by default is very important for newcomers. Like @tanadeau said, even experienced devs coming from dynamic languages like JavaScript, Python, Ruby or even static, enterprise languages like Java and C# are probably completely unaware of the concept of “debug” and “optimized” builds. Note that the above 5 languages represent the vast majority of the “first language” for people learning to code today.

rustc needs to be designed so that its users fall into the pit of success. Today this is sadly not the case; a newbie running rustc foo.rs will get a slow build by default, and might easily conclude that the equivalent code they wrote in Java is faster and that Rust can then be discounted. This isn’t a theoretical concern; situations similar to this come up on IRC all the time.

Anyone with more that passing experience with Rust will know how to get a debug build if they need one, but the default should certainly be an optimized build.

Think through why the unoptimized build is the default: the reason is convention started many decades ago by ancient compilers that didn’t have an optimized build at all, so when one was added in the future, it was added behind a flag one had to pass so as not to break the way people were using the compilers already. I don’t think anyone can make a reasonable case for “unoptimized by default” in a vacuum where no previous precedent is set by other compilers.

rustc should break with convention here because in this case, the conventional approach is hurtful to users. Today’s default is “fail-deadly” instead of being “fail-safe”; you forget to pass a flag you might not even be aware of and we give you a build that’s very slow and almost certainly not what you wanted (and if you’re a newbie, then it’s “certainly”, not “almost certainly”).

[tanadeau]

@Valloric Very well explained. I completely agree.

I should also mention that sometimes this can affect non-newbies as well. I’ve programmed in C++ for years professionally and on my own time. Given that, I also write a lot of Java and Python, and sometimes I’ll go back and forth from python foo.py to clang++ foo.cpp; ./foo and then have to go back and run clang++ -O2 foo.cpp; ./foo.

Perhaps I have a different workflow than most, but I spend more time running and benchmarking than needing to have the most info in a debugger session.

[tbu]

I think optimizing by default will shed bad light on Rust, because the edit-compile loop will be longer with this hypothetical change.

[tanadeau]

@tbu While that’s a valid concern, I think there are a few reasons the edit-compile loop should be of secondary importance:

1. Turning optimization on can often create a 10x perf increase while not taking 10x longer to compile. I think that makes it better from a cost/reward perspective.
2. I don’t think people care much about compilation time unless it’s taking more than a few seconds. For small programs, it’s not that big of a difference. By the time, compilation times would get very long, the user would have an investment in the language and hopefully have had a good experience with performance. That’s much better than someone writing a for loop, seeing slow perf, and never using Rust again.
3. Related to the above, compilation times are almost certainly never going to be a selling point for Rust. We’re probably never going to have compilation times as fast as Java or Go and definitely not the zero compilation times of an interpreted language. People are going to come to Rust due to safety, performance, and/or the systems features, at least at the start.

It’s definitely a trade-off, but I think it’s worth it.

[iopq]

But I compile way more than I actually run my code. You know, to make sure the program type checks after my changes? I do cargo build every atomic change I make.

I’m also a little bit afraid that if I arrow up twice and hit git reset --hard HEAD I’ll actually lose my changes instead.

[tomjakubowski]

The edit-compile cycle is already too long, because an inordinate amount of time is spent after the analysis phase even in unoptimized builds.

I’m apathetic about this particular issue, but I do think Cargo ought to support something like cargo check that runs only phases 1-3 (parsing, expansion, and analysis). In that case, at least the edit-compile cycle is unaffected by this proposal.

UPDATE: opened a Cargo issue for this: https://github.com/rust-lang/cargo/issues/1313