Trying to make Live++ support Rust

tools and infrastructure
1

Hello,

I am the author of Live++, a generic hot-reload tool for C & C++ for Windows (https://liveplusplus.tech), and I would love to add support for Rust to Live++. A few months ago I did a very quick prototype to show that hot-reload for Rust is doable and feasible, but had to hack something together using a custom Clang & LLD build: LiveRust prototype

Back then, the most important pieces missing in Clang were support for the S_OBJNAME (https://reviews.llvm.org/D43002) and LF_BUILDINFO (https://reviews.llvm.org/D80833) records in the PDB, which are needed by Live++ to know how to recompile individual object files. Since support for these records has landed in Clang 14, I thought I'd give it another try to see how far I'd get using Rust 1.60.

There are still a few minor road blocks preventing me from supporting Rust at the moment, so I hope any of you can help me with a few questions I have:

  1. Live++ needs the binary to be built with /FUNCTIONPADMIN, /OPT:NOICF, and /OPT:NOREF. I managed to add /FUNCTIONPADMIN and /OPT:NOICF by using a build.rs script, but it seems that /OPT:REF is always used by rustc internally. Is there any way to turn off /OPT:REF or override it?

  2. Live++ also needs object files to be compiled using the /hotpatch flag, which is always enabled by MSVC when building for x64, and has been added to Clang 14 (https://reviews.llvm.org/D116511). However, it seems that the rustc compiler doesn't understand this. Do you have any plans to support this for rustc as well? All it does (or needs to do) is making sure that the first emitted instruction in any function is at least 2 bytes long.

  3. Live++ works purely on a binary file, using the PDB and EXE/DLL files, object files, and static library files. It doesn't care about build systems, the source code, or anything really, but gathers the needed information from these files. As such, it also needs the object files generated by the compiler. However, when using "cargo build" on a simple hello_world, it seems as if individual .o files are first compiled into e.g. "target/debug/deps", linked into the binary, but then moved/renamed/deleted, ending up in the "incremental" folder. The problem with this is that the PDB module streams reference these .o files in their original location "target/debug/deps", but they aren't available there any more. This makes it impossible for Live++ to find these object files. Is there any option I can use to make cargo leave these .o files in their original location and not move them?

From the above, it's really not much that's missing to have hot-reload for Rust using Live++. I would love to hear your input on this, because it doesn't seem like we'd have to do any custom Live++ support in Rust in order to support it, but maybe just a few additional options in cargo/rustc instead.

9 Likes
2

One issue you'll probably hit is that the stdlib doesn't (currently) support being unloaded, and in particular unloading libstd will generally cause it to attempt to free thread locals when the thread ends, regardless of whether or not that code still is loaded. This has fairly bad side effects.

CC @chrisd, as we had a conversation about this in the recent past, and they have more context for the Windows side of things, which seems to be what you're interested in.

3

Live++ doesn't unload anything, so this is not a problem. It supports .exe and .dll just fine, and with .exe there wouldn't be anything to unload.

4

Interesting, nevermind then. I thought that the process of reloading included unloading and then loading again.

5

Bjorn3 from the Rust Community Discord server was able to provide more information. Using -Csave-temps will keep the object files in their original location, and -Clink-dead-code has the same effect as what OPT:NOREF does internally, however, it will still pass OPT:REF to the linker, which makes Live++ complain (but can be worked around for now).

6

That's a very cool functionality!

Rust users typically don't use the compiler directly, and don't work with object files or compiler flags. Rather, Rust is typically used via Cargo that only exposes a concept of crates and higher-level config. Since Rust/Cargo aim to offer strong stability and portability, changes you need may require individual consideration. Some harmless details of interaction with LLVM and debug info may be easy to change, some things that could be breaking or require new public interfaces (including UI and config) may require more consideration how to do them in a way that the Rust project can commit to supporting them forever, and not be a one-off hack for MSVC.

So I think it'd be best if you split your requirements into individual bugs/feature requests, and file them at https://github.com/rust-lang/rust/

1 Like
7

Thanks @kornel. We have moved the discussion to Zulip, where it got more attention: rust-lang

We're still in the phase of figuring out what exactly has to be done. I don't think we need any Live++- or MSVC-specific stuff in Rust for it to work.

2 Likes
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.