Pre-RFC: Remove Rust's dependency on Visual Studio in 4 (...complex?) steps

Feature Name: Darkroom (cos it's a studio ...but you can't see anything)

The following information are from a combination of somewhat scarce docs and own research. It’s known to be incomplet and incorrekt, and it has lots of bad formatting

Motivation

To build a Rust program for native Windows ("MSVC toolchain"), one needs to download Microsoft C++ Built Tools which is a part of Visual Studio and therefore require a Visual Studio licence. It is currently impossible to compile Rust for native Windows without a VS licence or without embedding (embedding, not just dynamically linking) proprietary Microsoft code.

We would like to avoid this such that a VS licence is no longer required to compile Rust for Windows MSVC. We'll also potentially enable people to cross-compile for native Windows on other platforms.

Summary

Diagrams rendered in GitHub MD with text

The Rust standard library depends on several components from Microsoft, this is where we are at now (graph stolen & adopted from Microsoft's C++ STL repository, Apache License v2.0 with LLVM Exception):

This is what we need to do:

This is the end result. Windows SDK is available on its own for free with a far less restrictive licence. In any case we have to use it to interact with the OS:

Below are a summary of things that I think need to happen before we can get to the :sparkles: shiny future :sparkles:. I am very much not an expert in this topic so please point out as many inaccuracies as you can find!

Explanations

Use lld

By default Rust uses link.exe on Windows, which is Microsoft's proprietary linker shipped with Visual Studio. But rustup already ships rust-lld and can be used by configuring .cargo/config (NOT Cargo.toml). There is already ongoing effort and progress to make LLD the default on Windows: Use lld by default on x64 msvc windows · Issue #71520 · rust-lang/rust · GitHub.

.libs Rust pulls in from Visual Studio

You can pass in RUSTFLAGS=--print=link-args to see the commandline used to invoke the linker. Overall, the Rust std on Windows pulls in the following libraries:

Library Rust std dependent VS or Win SDK Additional Compiletime dependencies Runtime dependencies on top of base Windows Imported Symbols from VS
advapi32.lib std::sys::windows Windows SDK None None
userenv.lib std::sys::windows Windows SDK None None
kernel32.lib std::sys::windows Windows SDK None None
ws2_32.lib std::sys::windows Windows SDK None None
bcrypt.lib std::sys::windows Windows SDK None None
msvcrt.lib Through libc crate VS ucrt.lib & vcruntime.lib None (Note: this is NOT the import library for msvcrt.dll) None, the lib exports no symbols, it only contains CRT initialisation and termination code.
ucrt.lib Pulled in by msvcrt.lib Windows SDK None UCRT (Shipped in base Windows since 10)
vcruntime.lib Pulled in by msvcrt.lib VS None Visual C++ Redistributable _CxxThrowException
__C_specific_handler
__CxxFrameHandler3
__current_exception
__current_exception_context
memcmp
memcpy
memmove
memset

There are two libraries exclusively shipped with VS: msvcrt.lib and vcruntime.lib which we need to remove as dependencies. They are both parts of "VCRuntime" in Microsoft's terminology even though its subcomponent is also called vcruntime.

The source code of VCRuntime is shipped with VS, available under C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\[version]\crt\src\vcruntime. But if you want to reimplement any of its functions you are strongly advised to refrain from reading it yourself because this code is "All rights reserved". The intention may have been to make it more difficult for anyone to claim that their reimplementations are clean-room because they cannot prove that they haven't seen the source code. This technique was used by a company where they printed their source code in the manuals, so anyone trying to reimplement their software would've likely seen the source code - I read this story somewhere but couldn't find it again, please comment if you know what I'm talking about!

Using compiler-builtins for memory functions

The Rust std depends on 4 primitive memory manipulation functions memset, memcpy, memmove, and memcmp (but not memchr because this is implemented in core). Currently the std gets them from libc crate which links against msvcrt.lib and in turn vcruntime.lib. These functions are also provided by the compiler-builtins crate so we could use them instead.

Note that the libc crate exposes virtually all standard libc functions, but the vast majority of them can be found in ucrt.lib so it's really only the above 4 we need to replace.

Write our own entry point

This section is currently only applicable to executables. I haven't looked into libraries yet

The Windows entry point function is called mainCRTstartup by default. However, this function is not generated by Rust. The first function created by rustc is simply called main (which is not the pub fn main() you wrote).

If you compile a Hello World program and decompile it with dumpbin /DISASM, then you can see it calls two functions __security_init_cookie and __scrt_common_main_seh, which are statically linked from msvcrt.lib.

__scrt_common_main_seh calls main generated by the Rust compiler, which then calls lang_start -> some Rust panic wrappers -> crate::main. Prior to calling main, the entry point needs to initialise all the global objects if no one else has done it (? who else can do it). It walks through a list of function pointers of C and C++ initialisers and calls them. The function pointers can be located through linker subsections. It also initialises TLS variables. Interestingly the global object initialisation is synchronised using a pointer-sized variable __scrt_native_startup_lock compiled into msvcrt.lib, but I don't think this is ever shared with another process because msvcrt.lib is always statically linked.

We need to write our own mainCRTstartup (we could call it something else too). I don't think we need any of the global object initialisation stuff so unless UCRT needs it then it should be fairly bare bones.

GS cookies :cookie:

__security_init_cookie unsurprisingly initialises the security cookie, which is used by Windows to detect buffer overruns. The source of this is shipped with VS but proprietary, but all it does is setting the global variable __security_cookie to a random value if it hasn't been initialised (it hasn't been initialised if its value is the default which are 3141592653589793241 >> 16 on x86_64 or 3141592654 on x86), and the new random value must not equal to the default. Another variable, __security_cookie_complement is set to the bit-wise complement of __security_cookie.

Rest of the :owl:: unwind our own stack

The other functions Rust std depends from vcruntime.lib are to do with exception handling, specifically Structured Exception Handling, which is a Microsoft C/C++ extension. Rust does not have exceptions, of course, but we do need to unwind the stack during a panic. I knew very little about stack unwinding but from the comments in std source code I think that we are calling _CxxThrowException on panic, Windows will then unwind our stack until it reaches Rust's landing pad.

We have two choices here: either ditch SEH entirely and implement our own stack unwinding mechanisms, or keep letting Windows unwind our stacks but clean-room reimplement the functions that triggers and catches the exceptions.

Alternatives

Ask Microsoft to release VCRuntime under a non-restrictive licence so we can compile & ship our own msvcrt.lib and vcruntime.lib

Unresolved questions

  1. Do we need to do any SEH setup for UCRT or does it do it on its own?
  2. How much CRT startup and termination do we need to do for UCRT, or does ucrt.dll handles it on its own?
  3. How does Go deal with interacting with UCRT? Or does it avoid it entirely?
6 Likes

A number of Rust programs depend on C++ libraries, and a number more may make use of the various crates that allow for pre-main code, so I would expect you’d want to implement that eventually, even if you didn’t worry about it right away.

1 Like

Ultimately, if you're linking against MSVC toolchain compiled binaries, you need to use (and have a license for using) MSVC. This is just a fact of using the MSVC toolchain on Windows.

Just clean room reimplementing the ABI isn't sufficient either, because Windows is perfectly able to adjust the ABI so long as their compilation tooling handles it. They do have plans for a C++ ABI break to happen eventually (though when is extremely unclear).

If you aren't using MSVC libraries, you can just already use the existing alternate toolchain: the pc-windows-gnu target triple. The pc-windows-gnu toolchain already works out-of-the-box on Windows without any MSVC tooling; the only limitation is that it can only use other libraries using the gnu ABI rather than the MSVC ABI.

(There was a period of time when VSWhere could not keep track of my Visual Studio installation where I used pc-windows-gnu as my main development target. I had no issues using it with pure-Rust code.)

2 Likes

The ideal plan isn't to reimplement the ABI but to avoid using any symbols in VCRuntime, then ABI breaks won't affect us. We'll still be linking to Win32 API because the import libraries are provided in Windows SDK.

This was talked about in RFC2627, though I don't think we actually need #[link(kind="raw-dylib")] to do what I proposed in this post.

With the features described in this RFC, we would be one step closer towards a fully standalone pure Rust target for Windows that does not rely on any external libraries (aside from the obvious and unavoidable runtime dependence on system libraries), allowing for easy installation and easy cross compilation.

  • If that were to happen, we'd no longer need to pretend the pc-windows-gnu toolchain is standalone, and we'd be able to stop bundling MinGW bits entirely in favor of the user's own MinGW installation, thereby resolving a bunch of issues such as rust-lang/rust#53454.
  • Also with that pure Rust target users would stop complaining about having to install several gigabytes of VC++ just to link their Rust binaries.

Implementation of a pure Rust target for Windows (no libc, no msvc, no mingw). This may require another RFC

This would have to be a separate target. The *-pc-windows-msvc targets allow using non-rust libraries that are compiled against vcruntime. If the vcruntime is not used, nor replaced with an abi compatible one, it would be impossible to use said libraries and thus we need to use a separate target for the target without vcruntime.

3 Likes

SEH can't be ditched, technically, as it's a Windows x64 ABI requirement to have exception descriptor tables for functions so the OS can unwind through your code (when debugging or someone explicitly calls the backtrace API). Fortunately, LLVM already handled all that.

Again, in theory, what vcruntime is providing here is an exception personality. I have read this code, and I can quite certainly tell you that is never going to be clean room implemented. The question is if it needs to be, the so far as I know, that would be so you can catch a msvc++ exception by type: pretty sure that's completely out of scope. So it would be interesting to see how much you can dummy out the exception_personality lang item: I think at least some of it it determined by LLVM emitted code? I certainly never got anything useful working, but there's vanishingly little documentation.

I'm curious to know where all this "technically" becomes "practically" though, to my knowledge the only interaction from Windows APIs with SEH is that any exceptions thrown from a window message procedure are caught and ignored(!) - and you can't catch exceptions from a different language anyway.

3 Likes

Yes, LLVM changes stuff depending on the exact name of the personality function on Windows, so we have to use the name of one of the C++ personality function versions AFAIK. It even seems LLVM asserts that the personality function is one of two names on SEH: llvm-project/X86WinEHState.cpp at 09c2b7c35af8c4bad39f03e9f60df8bd07323028 · llvm/llvm-project · GitHub If this personality function is not compatible with C++ it would be impossible to use C++ code in rust projects.

Longjmp is implemented using SEH forced unwinding. Furthermore pretty much everything that would be a signal on Unix is a SEH exception on Windows, so a segmentation fault will unwind the stack. Another thing is that things like a debugger or ETW tracing will not produce correct stacktraces in the absence of SEH unwind info. It also seems that SEH exceptions can be thrown for things that are normally recovered transparently by the environment. I didn't really find any concrete examples though.

3 Likes

Surely there are equivalents for these somewhere in kernel32.dll? (with different names) I'm not on Windows right now or I would check myself.

memset, memcpy, memmove and memcmp are also implemented by compiler-builtins on targets where they are not available from libc. Not all of said functions are as optimized as your standard libc though.

I think it'd be worthwhile to look into this given that Microsoft is investing heavily in Rust and there are folks from Microsoft involved in the Rust foundation. Perhaps @rylev or @nellshamrell could point you in the right direction?

5 Likes

Do you mean existing C++ libraries built against vcruntime would not have the personality function they expect available at link? Or that C++ code built as part of a crate would need to use the same personality?

Either way, throwing across FFI is still UB, AFAIK, even if Windows requires the support for it in the ABI, so I think it's probably fine to punt on C++ code and say that "you have to supply all of your C++ compiler, including a runtime", and Rust can do its own thing? Though it sounds like LLVM would need some changes.

Both

extern "C-unwind" may be stabilized in the near future. Throwing across extern "C-unwind" is fully allowed and in fact the reason this ABI has been introduced.

Hmm, presumably that still doesn't support catching across languages though? I'm a lot less confident here, but my understanding is multiple personalities works so long as they all check they have the right exception type, and pass it on otherwise.

Indeed. Only unwinding through is supported. Catching a foreign exception in rust is not UB though. It will simply result in a message being printed and then aborting. Having a separate personality for rust vs C++ would be fine if LLVM supported it in the first place. We already use a separate personality function with DWARF.

1 Like

It looks like Microsoft was already planning on open-sourcing vcruntime and vcstartup: Q3 2021 priorities · Issue #2046 · microsoft/STL · GitHub, though this is delayed and we don't have a new timeframe

Edit: got a response from the STL:

@cbeuw It's taking longer than we initially expected, but we're still working on it. We need three approvals - we've gotten one, @strega-nil-ms is working on adding benchmarking which will help with obtaining the second approval, and then we need final business approval.

7 Likes