Need help with emscripten port

llvm-config can do that:

~/PATH_TO_LLVM_BIN/llvm-config --targets-built

will return a space-separated list of targets, like:

X86 JSBackend WebAssembly

I’m not exactly sure how to get that into the rust build process :\


Some updates. I’m hoping to push emscripten support across the finish line in the next 2 or so months. That is, for the project to distribute emscripten-targetting compilers and stds (whether they contain awful bugs is another matter).

The short term plan is for kripken to port emscripten’s LLVM branch forward to the same merge-base as ours, then for us to rebase onto their branch. We don’t have any immediate plans to upgrade LLVM, but in the future we’ll have try not to do it too often, because each time will force emscripten to upgrade as well. Likewise we’ll have to upgrade when they do, but they don’t anticipate needing to either. And we’ll have to tolerate temporary breakage of our emscripten support if it gets difficult to do it simultaneously. It could end in disaster. Who knows.

Once we’re on their LLVM, distributing emscripten bins is mostly a matter of turning some knobs on the CI.

In the longer term, kripken is very keen on creating a MIR->wasm backend for Rust, bypassing LLVM. If it works out then it could open up a lot of really interesting possibilities.

@tomaka is the best way to compile for emscripten still to use my ‘emscripten’ branch or do you have something more up to date? I’ll aim to rebase it onto Rust master soon.


The Reddit article that was posted a few months ago was slightly more up to date, but not by much.

Or it could end in success! I’m super happy you all are trying to make this work. Thanks for all the effort!

It turns out that although our LLVM fork is further ahead than emscripten’s chronologically, that they are actually ahead of us logically. We are on 3.8 and they are somewhere in 3.9. So we’re going to have to upgrade. Unfortunately, this means we have to switch from make to cmake for building llvm, which is going to take some unusual effort. I’m going to put it on my todo list.

We’re using CMake now to build LLVM. So the path is clear to rebase the Rust and emscripten ports to the same merge-base. The plan is going to be complicated by the upcoming AVR port, which also wants to impose some LLVM churn.

Right now I think the best thing to do is for Rust to port forward to LLVM master (which we’re going to need to do no matter what), then coordinate with @shepmaster to figure out how much more upgrading we need to do on AVR, and @kripken to rebase onto the same commit.

Anybody interested in doing the LLVM forward port?

cc @alexcrichton

I’d really like to help with making this happen, though I have no deeper knowledge about the inners of rustc or LLVM.

As a potential starting point, I have some hacked up commits that might at least serve as guideposts for where Rust needs to be updated to accommodate LLVM 3.9.

@jer It doesn’t require deep knowledge of either usually to just get rustc building with a new LLVM, mostly just requires getting the Rust LLVM wrappers and src/rustllvm to build again. Once the new rustc builds you might run into test suite failures, and resolving those gets more difficult. If you wanted to get started you could just rebase the src/llvm submodule onto upstream llvm/master and start building it. @shepmaster’s fork should provide some solid hints about what needs to be fixed.

Thanks! I’ll take a look at it this week.

1 Like

As a note, my PR was closed because of inactivity but the changes are still needed to make emscripten work correctly.

So I took @shepmaster’s changes, applied them to a recent Rust checkout, checked out upstream master for llvm and compiler-rt and was able to compile stage 1 (full build is currently running).

@tomaka: I’ll take a look at that when I have the LLVM thing running.


Beautiful! Thanks for tackling this.

Current status: I got a full succesfull build, but make check fails in 4 cases (3 times LTO, where it can’t find main, it might be optimized out?). I really only took the existing changes from shepmaster and applied them, code lives here:

I hope to get around to clean it up next.

@jer Awesome! Only 4 failures is really encouraging. @kripken is about to start forward porting emscripten’s port to the same merge-base as yours.

Down to one actually, because I partly understood what I’m doing now. @alexcrichton said I should submit this as a PR, so it gets seen by a wider audience to help out fixing last few bits. I will get around to do this this week.


@jer has submitted the LLVM upgrade PR. @habnabit did some cleanup on my emscripten branch of Rust. It looks though like emscripten hasn’t yet updated the incoming branch of their fastcomp LLVM fork.

Next steps are to merge the LLVM upgrade, get @kripken to finish upgrading fastcomp, upgrade my ‘empscripten’ branch of Rust, port our LLVM patches onto fastcomp and submit that as a Rust PR, submit patches to turn on emscripten support in Rust, set up automation to keep emscripten target building.


The LLVM upgrade has now landed

Update: The Rust LLVM upgrade finally landed (thanks @jer and everyone who helped - it was a brutal slog).

Next @kripken is going to upgrade the emscripten incoming branch to the same merge base. But we don’t need to wait for that probably. He’s already prepared a rebase that should be in the ballpark of the same commit: the fastcomp next-merge branch.

So our next step is to take all of their patches, apply them to our tree and merge that into Rust.

Unfortunately, there’s a gotcha - @kripken didn’t realize that our plan was to apply the emscripten LLVM fork to all of our targets and he’s a bit worried about it. The reason is that the fastcomp branch is only tested for the emscripten target, and it’s based on the old pnacl fork, which has a bunch of ancillary patches of unknown value. So we may run into unanticipated problems (and in fact last time we had this working the x86_64 backend miscompiled things).

Furthermore, the fastcomp history is lengthy and messy, with all that pnacl legacy. So we are going to need to scrutinize it carefully and see e.g. if it’s doing anything to the x86 backend. We might, e.g just squash their entire history into one ‘emscripten’ commit, then revert large chunks of it. It could be a big mess. (We also have the option of pursuing a strategy where rustc can optionally use an emscripten-specific LLVM).

It’s also worth noting that the upstream LLVM wasm backend is still in progress and the last I heard the wasm team is trying to have it ready for the wasm launch late this year. So the longer this process goes on the more likely we are to just end up on the upstream wasm backend and skip the asm.js backend completely. Still, I think it’s worth continuing on the path we’re on and seeing how far we can get.


OK, another tantalizing avenue of attack here.

The LLVM wasm backend, while it isn’t fully working, and there are still changes to the wasm spec incoming, works to a relatively large degree, and Rust has it in tree now.

It would be very interesting to see what happens if we turn it on, start laying the groundwork for a wasm-unknown-emscripten target.

Emscripten already has experimental support, documented here. Might be some useful leads there.