Until now, rustc implicitly linked the jemalloc memory allocator to every Rust program (not staticlib or cdylib libraries) by default on some platforms (including Linux and macOS). jemalloc is often more performant than the system’s allocator, but not always, and it adds a non-trivial amount to the size of the executable.
In Rust 1.28, we stabilized the #[global_allocator] attribute that allows a program to opt into another memory allocator, and the std::alloc::System type to represent the system’s default allocator.
PR #55238 has landed today, removing jemalloc from the Rust standard library. So starting with tomorrow’s Rust Nightly, Rust programs will use the system allocator by default. You can use the jemallocator crate together with #[global_allocator] if you want to keep using jemalloc.
Changing the default took a long time because it was tricky to figure out a way to do it while still using jemalloc in rustc and LLVM-in-rustc, because of the unusual way rustc is linked. See issues #36963 and #51038 for more background. jemalloc is a significant performance improvement in almost all cases for rustc’s workloads.
FYI, the jemallocator crate makes using jemalloc as the global allocator as easy as adding it as a dependency and then setting it as the global allocator:
Just keep in mind that jemalloc does not support all targets that rustc supports (it supports many though), and that until now rustc implicitly selected a different allocator if jemalloc was not available for a target. Now you will have to explicitly fall back to the system allocator in those targets by #[cfg()]jemallocator away.
Perhaps the jemallocator readme can keep an updated #[cfg] for the targets it supports so it’s easy to copy/paste that without failing to compile on random platforms?
@ahmedcharles I wanted to add a crate to the jemallocator repo that just sets jemallocator as the #[global_allocator] for you, so that the only thing one has to do is add that crate as a dependency and “that’s it”.
We can set that crate up so that it does nothing on platforms that are not supported by jemalloc (e.g. by being empty), and add an opt-in feature to it that produces a compile_error! instead.
That would allow you to choose whatever behavior you want with little effort, and either just fall back to the system allocator, or add some other crate specifying some other #[global_allocator] on those platforms without collisions (there can only be one #[global_allocator] in a program).
@SimonSapin@Sfackler do you think that something like this might be a good idea?
The compiler is the one that selects which allocator is in use, and deep within it you’ll find logic that looks for #[global_allocator] and, failing that, will use some symbols defined in the standard library instead. It’s… a bit complicated and convoluted how this is implemented, but thankfully not too relevant to users!
Allocation APIs call some extern functions declarations with known symbol names. For “leaf” (non-rlib) creates, the compiler generates these functions to call either the extern function definitions generated by #[global_allocator], or the differently-named definitions in std.
wasm targets are not affected by this change. They use a Rust port of dlmalloc by default. I don’t know if anyone has tried to compile jemalloc’s C code to wasm an make it work in that environment.
This should make using valgrind with Rust a works-out-of-the-box experience!
Could you clarify why removing jemalloc makes it easier to run valgrind? I've been using valgrind for a while now. (It's how I noticed that the allocator changed! )
)