# Making Rust usable on a rootless, out of the box Linux/macOS system (no system linker) **URL:** \ **Category:** compiler\ **Created:** [June 10, 2023, 6:52pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966 "2023-06-10T18:52:53Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![cbeuw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cbeuw/32/7998_2.png) [@cbeuw](https://internals.rust-lang.org/u/cbeuw)\ **Post date:** [June 10, 2023, 6:52pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/1 "2023-06-10T18:52:53Z")
This may be a rather niche issue, but as an undergrad I had quite a few run-ins with systems on which I have no permission to use the package manager. Normally `gcc` is installed on Linux, though sometimes not (it's not a default package on most distros). On macOS, the build tools are generally not available and I have no permission to run `xcode-select --install`. For most software and language ecosystems, if I can't use the package manager, that's it, there's little hope to get it running. But Rust is _so_ close to runnable it's more frustrating when it doesn't: rustup, Cargo and rustc are all standalone executables with no external library dependencies, they are installable and runnable on a brand new install of Ubuntu/Debian/Fedora/macOS as a non-priviliged user **except** the system linker invoked through `cc`, for which one has to install with root privilege. You can't even "just" compile a copy of gcc or clang because you have no compiler (not to mention their other dependencies)! It would be really nice to be able to compile Rust program on a completely clean install of Linux or macOS, as this should be currently possible [unlike on Windows](https://internals.rust-lang.org/t/pre-rfc-remove-rusts-dependency-on-visual-studio-in-4-complex-steps/16708). AFAIK unless cross-compiling, pure-Rust programs don't need any header files or dynamic libraries that isn't shipped by default with standard Linux distros or macOS. `zig cc` is a [workaround](https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html), but in my testing it still breaks when supplied with certain flags (I wasn't able to build any dependencies with proc macros on macOS, for example). I don't know what is needed to reach the above goal, but I imagine this involves shipping a copy of `lld` through rustup like we currently do on Windows, and make this usable through `-Clinker=lld` (but not `-Clink-arg=-fuse-ld=lld` as we don't have `cc`). ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [June 10, 2023, 7:45pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/2 "2023-06-10T19:45:05Z")
Rustup does provide a copy of lld, as `rust-lld`. The main problem with attempting to use that as the linker is that it doesn't typically know about system library paths, so you are likely to get "unable to find library" errors. And the result may not run as expected. That's not a path for folks who want a seamless experience; that would take work to change. I _think_ people are looking at that but I don't know the status of it. There's been discussion over the years of making `rustc` capable of compiling C, similar to zig; there are people excited about that idea, but please note that there's no consensus on doing that, and there's no guarantee of it happening anytime soon, or at all. ---
**Author:** ![cbeuw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cbeuw/32/7998_2.png) [@cbeuw](https://internals.rust-lang.org/u/cbeuw)\ **Post date:** [June 10, 2023, 7:48pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/3 "2023-06-10T19:48:37Z")
> [@josh](#): > > Rustup does provide a copy of lld, as `rust-lld`. This is true on Windows but I cannot find `rust-lld` under a default nightly toolchain on Linux and macOS, not even with `llvm-tools` component installed. ---
**Author:** ![Nemo157](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/nemo157/32/11585_2.png) [@Nemo157](https://internals.rust-lang.org/u/Nemo157)\ **Post date:** [June 10, 2023, 8:03pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/4 "2023-06-10T20:03:15Z")
On Linux it appears to be part of the `rustc` component, installed at `/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld`. ---
**Author:** ![cbeuw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cbeuw/32/7998_2.png) [@cbeuw](https://internals.rust-lang.org/u/cbeuw)\ **Post date:** [June 10, 2023, 8:08pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/5 "2023-06-10T20:08:00Z")
Ahhh that's where it is. Though I think it still somehow attempt to invoke a system build tool ---
**Author:** ![Nemo157](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/nemo157/32/11585_2.png) [@Nemo157](https://internals.rust-lang.org/u/Nemo157)\ **Post date:** [June 10, 2023, 8:12pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/6 "2023-06-10T20:12:35Z")
I believe that path isn't added to the `PATH` by default, so you would have to either add it yourself or use an absolute path. There is a plan to add a new flag `-Clink-self-contained=linker` that will add it automatically, which combined with `-Clinker-flavor=gcc-lld` replaces the current `-Zgcc-ld=lld`: > > > \# Proposal > > In the context of improvements to compile-times, we'd like to make… progress on \[enabling LLD by default\](https://github.com/rust-lang/rust/issues/39915). Since this is a broad topic, with known issues and multiple stakeholders, the goal is to first focus on an achievable subset -- \[enabling LLD by default on linux to start\](https://github.com/rust-lang/rust/issues/71515) -- and to make incremental progress towards that in multiple steps. > > This MCP is proposing such a first step, stabilizing the \`-Zgcc-ld=lld\` flag to be able to use \`rust-lld\` (avoiding the known \`lld\` issues for now, allowing to focus on a smaller scope and make progress), and then other follow-up tasks towards achieving the goal. > > The system's \`lld\` can already be used on stable, and we build and distribute \`rust-lld\` in \`rustup\`. Using \`rust-lld\` would alleviate the need to install \`lld\`, and possibly avoid some incompatibilities between the system version and the LLVM version used by rustc (or the need to keep them in sync). On some targets the wrappers and executable can be used directly, but it's not the case by default on linux, and that's where \`-Zgcc-ld=lld\` currently helps. > > \[\`-Zgcc-ld=lld\`\](https://github.com/rust-lang/rust/blob/e7575f9670f3c837def3d186ae09366c75c7632e/compiler/rustc\_codegen\_ssa/src/back/link.rs#L2572-L2614) uses \[\`lld-wrapper\`s\](https://github.com/rust-lang/rust/tree/master/src/tools/lld-wrapper) to call \`rust-lld\`. They are compiled as \`ld\` and \`ld64\` binaries, and distributed in a \`gcc-ld\` folder in the sysroot next to \`rust-lld\`. The flags makes sure the wrappers exist, and adds arguments to the \`Command\` used to do the linking. > > \--- > > \#### Stabilizing \`-Zgcc-ld=lld\` as \`-Clink-self-contained=linker -Clinker-flavor=gcc-lld\` > > The details of this proposal were discussed in this \[zulip thread\](https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Stabilize.20.60-Zgcc-ld.3Dlld.60.20compiler-team.23510) and the discussion there has settled to stabilize a way to use \`rust-lld\` by splitting the 2 things that \`-Zgcc-ld=lld\` does more cleanly: > \- it requests using \`lld\`. We propose that this is done via a dedicated \`-C linker-flavor=gcc-lld\` option that would itself only add the \`-fuse-ld=lld\` link argument. (Or as a colon-separated \`gcc:lld\`, allowing for extensions such as passing anything after the \`:\` separator straight through \`-fuse-ld=\` and work out of the box for \`gold\` and \`mold\` if supported by the installed version of GCC) > \- it uses \`rust-lld\` instead, by passing the path needed to find the \`lld-wrappers\` within the sysroot. We propose this is done via a new option to an existing flag, a \`-Clink-self-contained=linker\` option. This flags currently only targets linking our CRT objects on a few targets. > > Currently, the same enums are used internally to handle the CLI's codegen flags, by the target specs themselves, as well as all linking related code. This can cause issues of duplication, and be error-prone: it seems some \`lld\` flavor variants have been created for CLI use, but they must be handled the same way internally. We propose splitting the surface enums used for CLI, so that the changes to the flag values don't change the internal linking code or target specs. > > \##### Testing > > Looking at the issues in the rust repo, I couldn't find some related to \`-Zgcc-ld=lld\` itself. There are a few about \`rust-lld\`, but on other targets. > > I have tested enabling it on \`x86\_64-unknown-linux-gnu\` with the 800 most popular crates on crates.io (most of them are libraries so linking and executing only happens for build scripts and proc macros) and a dozen popular binaries (cargo, ripgrep, nushell, tokei, etc) without obvious problems. It's possible there are still issues, in addition to the known issues about using \`lld\` in general: it's unclear whether this flag is well-known and used in the community, so a \`build-and-test\` crater run would be at least reassuring. I've opened \[PR 96025\](https://github.com/rust-lang/rust/pull/96025) to do a crater run with \`-Zgcc-ld=lld\` enabled. > > \##### Of note > \- distro builds don't bundle \`rust-lld\`, so \`-Clink-self-contained=linker\` should either be a no-op there, or produce a warning. This could be requested with a \`config.toml\` flag, either a new one or the existing \[\`rust.lld\` flag\](https://github.com/rust-lang/rust/blob/27af5175497936ea3413bef5816e7c0172514b9c/config.toml.example#L559-L561). > \- to allow for some experimentation and testing on nightly, the new flag values will be requiring \`-Z unstable-options\` in the beginning. > > \--- > > \#### Follow-up tasks > > These follow-up tasks could be next steps towards using \`rust-lld\` as the default linker on linux: > > 1) Tracking known issues when using \`lld\`, finding ways to fix them or have workarounds. > > There's an issue with stack traces generated when using \`perf\`, \[detected in \`flamegraph-rs\`\](https://github.com/flamegraph-rs/flamegraph/pull/157). This one doesn't seem to be tracked in the rust or LLVM repositories. The impact in practice is still a bit unclear (if it's not limited to the flamegraph use-case), and it doesn't seem to affect e.g. \`cachegrind\`. It probably should at least be tracked in our issues, since it could be decided to be a blocker. It seems unlikely, as there is a workaround (the \`--no-rosegment\` link arg) and we could imagine using it by default when \`rust-lld\` is enabled. > > There is \[one issue\](https://github.com/rust-lang/rust/issues/71515) related to coverage on the musl target, but this could be avoided and fixed later, by focusing on enabling it only for \`x86\_64-unknown-linux-gnu\`. This specific use-case would then use the existing default. (The workaround for the previous issue doesn't seem work in this case) > > Having LLVM/LLD experts look at these would be good. They could know whether these are issues in \`lld\` or our use, their severity, etc. > > 2) Likely publicize the new flag, so that users can try it out and report issues. > > 3) Eventually, depending on the results of the previous two tasks, discussing switching the default to \`rust-lld\`. > > \# Mentors or Reviewers > > Maybe @petrochenkov, since they've reviewed the \[PR adding \`-Zgcc-ld\`\]( https://github.com/rust-lang/rust/pull/85961) ? > > \# Process > > The main points of the \[Major Change Process\]\[MCP\] are as follows: > > \* \[x\] File an issue describing the proposal. > \* \[\] A compiler team member or contributor who is knowledgeable in the area can \*\*second\*\* by writing \`@rustbot second\`. > \* Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a \`-C flag\`, then full team check-off is required. > \* Compiler team members can initiate a check-off via \`@rfcbot fcp merge\` on either the MCP or the PR. > \* \[\] Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered \*\*approved\*\*. > > You can read \[more about Major Change Proposals on forge\]\[MCP\]. > > \[MCP\]: https://forge.rust-lang.org/compiler/mcp.html > > \# Comments > > \*\*This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.\*\* Once that's implemented I assume you could try to use it directly without `gcc`, but would then run into the issues josh mentions that together `rustc` and `lld` still don't know everything required to get a working binary. ---
**Author:** ![mathstuf](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@mathstuf](https://internals.rust-lang.org/u/mathstuf)\ **Post date:** [June 10, 2023, 8:20pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/7 "2023-06-10T20:20:40Z")
> [@cbeuw](#): > > On macOS, the build tools are generally not available and I have no permission to run `xcode-select --install`. Note that if Xcode is around, you do not need to install the command line tools (in fact, I explicitly do _not_ do so for our CI machines because we have a variety of Xcode versions available, but CLT is a unique system-wide resource and controlling its version reliably is not possible (AFAIK). Instead, you can get by with `export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer`. ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [June 10, 2023, 11:16pm UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/8 "2023-06-10T23:16:54Z")
> [@Nemo157](#): > > I believe that path isn't added to the `PATH` by default, so you would have to either add it yourself or use an absolute path. `-C linker=rust-lld` successfully references that (but then runs into all the issues associated with trying to use it). ---
**Author:** ![Nemo157](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/nemo157/32/11585_2.png) [@Nemo157](https://internals.rust-lang.org/u/Nemo157)\ **Post date:** [June 11, 2023, 8:38am UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/9 "2023-06-11T08:38:51Z")
Interesting. I wonder what `-Clink-self-contained=linker` is meant to do then 🤔, skimming the (closed) PR implementing it ([#96884](https://github.com/rust-lang/rust/pull/96884)) it looks like it parses it but doesn't actually change any behavior based on it. ---
**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\ **Post date:** [September 9, 2023, 8:39am UTC](https://internals.rust-lang.org/t/making-rust-usable-on-a-rootless-out-of-the-box-linux-macos-system-no-system-linker/18966/10 "2023-09-09T08:39:33Z")
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.