Moving WebAssembly support forward

[brson]

@tlively How close is your port to being feature complete? After thinking a bit more I’m a little concerned that it could be premature to upgrade llvm right now. I have the impression that the wasm backend and lld are not fully working yet, so it seems quite likely we will be in the position of doing another llvm upgrade in short order before users can really use the target. Do you think that would be the case?

[tlively]

@brson Yes, I think that would definitely be the case. I think it would be best for me to work locally and find any obvious bugs before we go through all the work of upgrading LLVM upstream. This will give @sbc100 time to upstream his work on lld as well. I think we will be in a better place to do the upgrade in about two weeks.

That being said, perhaps we can start an upgrade now, because subsequent upgrades would only be small incremental changes. That might be too much overhead, though.

[kripken]

Emscripten is a large dependency, and as LLVM tool support for WebAssembly matures, Emscripten will no longer be the best way to support compilation to WebAssembly.

I think that might be worth clarifying. Emscripten contains a bunch of things, one of which is fastcomp and the asm.js backend there. Maybe that’s what you mean by “emscripten” in that sentence?

Moving Rust to use the wasm backend can avoid the fastcomp dependency (which would certainly be nice!) but I’d recommend you still use emscripten to drive the wasm backend, as it provides a lot of things (system libraries, Web API integration, etc.) that otherwise you’d need to do all yourself.

Btw, while getting rid of the fastcomp dependency is nice, getting rid of the LLVM dependency might be even nicer which is what mir2wasm does. That should be even simpler than the wasm backend (fewer and smaller dependencies), and as a bonus should easily win on compile times. Would be nice to see experimentation in both.

Back on topic, for the shorter-term issue here, if rust wants to keep support for the asm.js backend while adding wasm backend support, then we’d need to update LLVM in fastcomp. I can assist there, but don’t have time to do the LLVM upgrade itself (unless this can wait a while until I do).

[vadimcn]

These will be tied to Emscripten’s LLVM updates as long as we still want to support asm.js and there is no wasm2asm tool.

Actually, it looks like there is. I don’t know if quality of the emitted code is comparable to emscripten, but maybe it’ll be enough to fill the gap for now?

Another thing we could do is to have a special build of rustc based on emscripten’s LLVM, while the regular build uses newer LLVM and supports asm.js via wasm2asm. Unfortunately, this would put even more load on Rust’s CI infrastructure.

[kripken]

These will be tied to Emscripten’s LLVM updates as long as we still want to support asm.js and there is no wasm2asm tool.

Actually, it looks like there is. I don’t know if quality of the emitted code is comparable to emscripten, but maybe it’ll be enough to fill the gap for now?

Sadly that wasm2asm tool is not in a usable state. It was an experiment which ran into some issues, and we’ve hoped someone would have time to work on it some more, but that never happened.

[vadimcn]

Moving Rust to use the wasm backend can avoid the fastcomp dependency (which would certainly be nice!) but I’d recommend you still use emscripten to drive the wasm backend, as it provides a lot of things (system libraries, Web API integration, etc.) that otherwise you’d need to do all yourself.

I agree, emscripten provides a lot of stuff for free. However, using emcc as a linker requires us to have bitcode-level compatibility with emscripten, which restricts our choice of LLVM version we can use. Things would be easier if emscripten grew support for accepting wasm modules as inputs. Is that feasible?

The other approach we could take, is to leverage emscripten at toolchain build time to create wasm versions of libcompiler-rt, libc and libm, but take care of linking the final executable ourselves (by directly invoking lld). This way end-users wouldn’t need to install emscripten at all, as long as they compile pure-Rust programs. Well, I am sure I am glancing over some important details here, like “Who’s going to emit the .js bootstrapper for wasm modules?”, so maybe we’ll need to include parts of emscripten too.

[kripken]

Moving Rust to use the wasm backend can avoid the fastcomp dependency (which would certainly be nice!) but I’d recommend you still use emscripten to drive the wasm backend, as it provides a lot of things (system libraries, Web API integration, etc.) that otherwise you’d need to do all yourself.

I agree, emscripten provides a lot of stuff for free. However, using emcc as a linker requires us to have bitcode-level compatibility with emscripten, which restricts our choice of LLVM version we can use. Things would be easier if emscripten grew support for accepting wasm modules as inputs. Is that feasible?

Yes, I think that’s the right approach, and sbc100 is working on it, see https://github.com/kripken/emscripten/pull/5313

[whitequark]

I’d suggest we name it rust-lld, distribute it with every host configuration (even when unused) and put it in /bin though I know @alexcrichton is inclined to put it in rustlib/$target/bin like we do with gcc on windows.

Why not link it into rustc, like we do with LLVM? Seems like it’s somewhat nicer to have the entire thing in one executable, for one, no issues with escaping, command line lengths, …

[Ericson2314]

Exactly. If we used stock lld, then that might make sense. But if we are linking the library anyways, yeah just skip the boilerplate of multiple processes.

If we do go the separate process route, it absolutely should not be in a target-specific directory as all things llvm are multi-target.

[vadimcn]

@kripken, is there a way to coax emcc into compiling its runtime libraries for wasm without giving it any source or bitcode files? I think I’d rather invoke lld for wasm objects myself, but would like to reuse the work you’ve done around runtime libs.

[briansmith]

Doesn’t the lld version need to match the clang version in order for clang’s LTO to work for C/C++ parts of a project?

[kripken]

@kripken, is there a way to coax emcc into compiling its runtime libraries for wasm without giving it any source or bitcode files? I think I’d rather invoke lld for wasm objects myself, but would like to reuse the work you’ve done around runtime libs.

You can use the embuilder.py tool to build system libraries. However, those are bitcode or wasm files, and you also need to build the bitcode ones into wasm, and also need to generate the JS runtime support for them. Emscripten support for wasm input files will make that part possible too, basically you’ll be able to build wasm however you want (e.g. using lld), then invoke emcc on the wasm to generate all the system lib and runtime support you need.

[vadimcn]

How do I use it embuilder?

“WASM=1 embuilder.py build wasm_compiler_rt wasm-libc” says “WARNING:root:wasm_compiler_rt not built when using JSBackend”

“EMCC_WASM_BACKEND=1 embuilder.py build wasm_compiler_rt wasm-libc” errors out:

CRITICAL:root:WebAssembly set as target, but LLVM has not been built with the WebAssembly backend, llc reports:
===========================================================================

    js     - JavaScript (asm.js, emscripten) backend
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64

===========================================================================

Also, where does it output the files it builds?

Emscripten support for wasm input files will make that part possible too

Any idea what’s the timeline for this? I’d like to experiment with wasm backend in parallel with lld’s wasm support being finalized.

[kripken]

How do I use it embuilder?

“WASM=1 embuilder.py build wasm_compiler_rt wasm-libc” says “WARNING:root:wasm_compiler_rt not built when using JSBackend”

I don’t think WASM=1 in the env has an effect, but maybe I forgot something (is it mentioned in the docs somewhere?). The warning is saying that that library is not needed when using the asm.js backend.

“EMCC_WASM_BACKEND=1 embuilder.py build wasm_compiler_rt wasm-libc” errors out:

CRITICAL:root:WebAssembly set as target, but LLVM has not been built with the WebAssembly backend, llc reports:
===========================================================================

    js     - JavaScript (asm.js, emscripten) backend
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64

===========================================================================

That env var means it tries to use the wasm backend, but the wasm backend is not built with emscripten’s default LLVM (since it isn’t stable yet). Instead you’d need to build LLVM (with that backend enabled, I think they don’t have it enabled by default, it’s experimental), and point emscripten to use that (e.g. by setting the LLVM env var).

Also, where does it output the files it builds?

It should say after it builds where it placed the outputs, they will be in the cache dir, usually something like ~/.emscripten_cache/...

Emscripten support for wasm input files will make that part possible too

Any idea what’s the timeline for this? I’d like to experiment with wasm backend in parallel with lld’s wasm support being finalized.

I’m not sure, might want to ask in that PR.

[Ericson2314]

Ah I didn’t think of that. I assume lld doesn’t link clang but just thebotcode must match? Do we need rust-clang and rust-lld?

[brson]

Why not link it into rustc, like we do with LLVM? Seems like it’s somewhat nicer to have the entire thing in one executable, for one, no issues with escaping, command line lengths, …

I like having in a separate process because for debugging it’s sometimes useful to drive the compiler and linker separately, but then again I’m rarely doing this kind of debugging myself, so if rustc devs want it to be in-process then I don’t care much.

[sanxiyn]

Note that we already run assembler in-process instead of running separately. In theory, -C no-integrated-as runs assembler separately although I am not sure how well it works… We could have -C no-integrated-ld too.

[tlively]

HI everyone, I just wanted to share an update on the status of this work.

Wasm support in lld is still not complete or stable, though it is much further along than before. As such, it hasn’t made sense to start the LLVM upgrade that would be a prerequisite for emitting wasm native object files from rustc.

I’ve also been thinking more about how to add in the JavaScript glue and libc support and everything else emscripten provides, and it seems to me that for users who need a full POSIX environment with a file system and everything else, emscripten will continue to be the most complete solution and we should still support it. As such, once lld stabilizes more, I plan to update wasm32-experimental-emscripten to emit native wasm objects files and throw those over the fence to emscripten, treating emcc like a normal linker that also happens to add any necessary JavaScript glue. This will make wasm32-experimental-emscripten work the same as any other native target, except that it will use emcc instead of cc for the final linking. I have this working locally, so once LLVM is upgraded this will be a quick update. Another benefit of this approach is that lld does not need to be in the Rust tree, since it will be distributed with and driven by emscripten.

However, I do still think that having the toolchain depend on emcc even for users who don’t depend on full POSIX support on the web is overkill. @vadimcn has been working on pulling the essential JavaScript libraries out of emscripten so they can be distributed statically as part of the WebAssembly rustc component. We don’t yet have a solid story for how or where to emit the JS glue, but I would like to see that functionality be as small and simple as possible. This type of support for wasm would require lld to be in the Rust tree in one form or another.

In the meantime, I am going to be working on getting the test suite to pass on the current wasm32-experimental-emscripten. Now that https://github.com/rust-lang/rust/pull/43175 has landed, you can run the disabled wasm-exp builder locally to see what’s passing or failing. Make sure to have experimental-targets = WebAssembly in the LLVM section of your config.toml if you try this.

[vadimcn]

FWIW, I concur with @brson: it is much easier to debug linking problems when you can run linker as a separate tool.

Also, what would integration of linker buy us?
In the case of assembler, the benefit is that we can avoid serializing in-memory representation of machine instructions into a text format only to have assembler immediately convert it back into a similar form. For linking, most of the inputs had already been serialized to files, so we wouldn’t be gaining nearly as much.

[brson]

Thanks for the update @tlively. I agree with your assessment that we should continue deferring to emcc for projects that want to target an emulated POSIX environment.

Tangentially, I’ve heard from @wycats that he needs to be able to generate no_std wasm binaries that are very minimal and contain no extra binary gunk that strictly necessary, and has had problems in the past doing so. ISTM that it should be possible to do a no_std build against emcc without pulling in any more of the emscripten runtime and glue than necessary, but I don’t know. A non-emscripten wasm target would also be good far that use case.