Rustc build memory requirement

compiler
1

Currently README says “Rust currently needs between 600MiB and 1.5GiB of RAM to build”, but it seems outdated.

I clearly remembered building Rust on 2 GB host, so I recently purchased Scaleway 1-S VPS which provides 2 GB memory. HEAD build swapped and eventually got killed.

So 1. what is the current build memory requirement? 2. Is it desirable to keep this under 2 GB? (I think it is.)

1 Like
2

If build requirements are to be standardized, I think that an amount of memory per CPU core would make more sense than an absolute amount of memory, due to the fact that current build systems (including cargo) use a process-based parallelization approach whose RAM requirements scale ~linearly with the amount of processes in flight.

The de facto standard for mid-end consumer machines (i.e. those with an Intel iSomething in it) is 2GB/core, that could be a good baseline. But if the 1GB/core that you are requesting is feasible in a straightforward way, it would be helpful for more resource-constrained environments, such as cheap cloud VMs, embedded chips, or low-end computers.

2 Likes
3

I believe I've seen a single rustc process using >4GB (for librustc in incremental).

I wonder if the compiler-performance working group could track RAM usage as well as CPU...

4

Is there a way to restrict rustbuild to a single core in order to avoid running out of memory?

5

Using GNU time, I measured 2.7 GB usage for clean build on x86_64-linux.

6

Is this something that would be platform dependent or not really ? E.g. armhf builds taking more/less memory thatn x86 ones ?

7

Notice: this got bad enough that rustc will be removed from Debian MIPS.

https://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/2019-January/004844.html

1 Like
8

Is that optimized+debuginfo? I’ve been disabling debuginfo in 32-bit rustc builds for Fedora. Having a “bare” build is better than nothing at all…

9

Is that optimized+debuginfo? I've been disabling debuginfo in 32-bit rustc builds for Fedora. Having a "bare" build is better than nothing at all...

isn't disabled by default ? here the config.toml.example provided by 1.32.0:

# All options are commented out by default in this file, and they're commented
# out with their default values.

[rust]
#optimize = true
#debug = false
#debug-assertions = false
#debuginfo = false
#debuginfo-lines = false
#debuginfo-only-std = false
#debuginfo-tools = false

I am not familiar with Fedora, could you provide a link to the patches list applied for Fedora packaging/distribution ? I would be interested to take a look (disclosure: I am the maintainer of lang/rust for OpenBSD).

10

Those config.toml defaults are only accurate with the default "dev" release channel. For stable/beta/nightly, you'll still get debuginfo and debuginfo-tools = false, but debuginfo-only-std and debuginfo-lines = true. This results in -Cdebuginfo=1 just for std and its dependencies, which shouldn't be a problem for memory usage.

For Fedora, the usual packaging policy is to build everything with optimization and debuginfo, overriding upstream defaults as needed. The rpm build will then strip those debug sections into separate files for a foo-debuginfo subpackage, so normal users don't incur any download/storage cost for this. I believe Debian does similar.

Fedora's package is here: Tree - rpms/rust - src.fedoraproject.org

I also just looked at Debian's package, and in config.toml they have:

debuginfo = true
debuginfo-lines = true
debuginfo-only-std = false

FWIW, I think the lines setting is redundant when full debuginfo is already enabled. You could look at a verbose build log to see what -Cdebuginfo level is actually used, where 1 is just lines and 2 is full.

And in their rules:

# Work around low-memory (32-bit) architectures: https://github.com/rust-lang/rust/issues/45854
ifneq (,$(filter $(DEB_BUILD_ARCH), armhf armel mips mipsel powerpc powerpcspe))
	sed -i -e '/^debuginfo-only-std = /d' "$@"
endif

Having deleted this setting, it should take the default true for stable. The largest culprits like libsyntax_ext and librustc should be without debuginfo in this case.

However, in Debian's latest mips log, it looks like those two crates are building at the same time. Maybe that's still too much for the system to bear? Have you tried with ./x.py build -j1?

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.