Questions about LLVM geneology, Wasm, Emscripten

eholk · 2017-05-24T22:30:43.656Z

Hi all,

I’m trying to make sense of Rust’s fork of LLVM.

My understanding is that overall Rust builds against a released version of LLVM, likely with a small number of Rust-specific changes. Because LLVM changes tend to break the Rust bindings, LLVM upgrades are a nontrivial undertaking, and thus happen somewhat infrequently. Is this correct so far?

Rust also has asm.js and Wasm targets, which use fastcomp, which is from a fork of LLVM. At first I thought this meant Rust would change LLVM branches when building the emscripten target, but it seems like actually Rust uses one branch that has either merged in or cherry-picked fastcomp. Is this right?

Next, the Wasm LLVM backend seems to have advanced somewhat since January when it seems Rust’s LLVM was last significantly updated. Would the Rust team be open to updating the LLVM Wasm backend? If so, would it be better to do it with cherry picking, merging, subtree merges, or trying to do another LLVM upgrade?

brson · 2017-05-24T23:58:03.925Z

It looks like this is our fork. There was a time when we only carried a few patches, but it looks quite monstrous at the moment.

Rust does not base it’s fork off of a released version of LLVM. It is an arbitrary commit from upstream LLVM. LLVM upgrades do break Rust’s bindings but that is usually not the hardest part of the upgrades. Rather, getting all of Rust’s supported platforms to pass CI again is. Our MinGW port is especially troublesome, as upstream support has degraded over the years.

It looks like we are currently carrying a merge of the fastcomp fork, though when I originally merged fastcomp support, we did not literally merge with fastcomp, we just created a minimal patch that created a fastcomp target with fastcomp-compatible IR.

If we try to pull in an updated wasm backend I’d prefer to upgrade LLVM from upstream and not cherry-pick more stuff.

cc @TimNN did the last upgrade.

brson · 2017-05-25T00:01:09.237Z

If we do an upgrade, then chances are emscripten will need to as well.

eholk · 2017-05-25T00:24:02.319Z

brson:

If we do an upgrade, then chances are emscripten will need to as well.

Hmm, does that mean that Rust is now basically unable to upgrade LLVM unless Emscripten also does so?

brson · 2017-05-25T02:24:01.064Z

eholk:

Hmm, does that mean that Rust is now basically unable to upgrade LLVM unless Emscripten also does so?

Yes, that is the case.

droundy · 2017-05-30T20:42:54.029Z

This all sounds kind of crazy. Is upstream LLVM really so unstable that it regularly breaks platforms? Or is it that rust is using LLVM in ways that LLVM wasn’t designed for?

steveklabnik · 2017-05-30T21:02:27.267Z

LLVM puts out a new major version roughly every six months, so it’s not super super stable.

brson · 2017-06-02T18:07:42.511Z

droundy:

This all sounds kind of crazy. Is upstream LLVM really so unstable that it regularly breaks platforms?

Yep. LLVM supports a lot of platforms and architectures, with greater and lesser degrees of support and testing. Apple platforms work great!

brson · 2017-06-02T18:10:41.726Z

It could be that basing our fork off of arbitrary commits causes us more platform regressions than using an LLVM release. Presumably they catch some regressions ahead of release.

eholk · 2017-06-02T18:12:53.805Z

Out of curiosity, what all is in the Rust LLVM fork that’s not in regular LLVM? Are these changes still necessary, or would it be feasible to consider moving back to a vanilla LLVM release?

brson · 2017-06-02T18:34:02.370Z

Just going through the list of patches it looks like these are not from upstream:

https://github.com/rust-lang/llvm/pull/73 https://github.com/rust-lang/llvm/pull/70 https://github.com/rust-lang/llvm/pull/45 https://github.com/rust-lang/llvm/pull/49 https://github.com/rust-lang/llvm/pull/34 https://github.com/rust-lang/llvm/pull/62 https://github.com/rust-lang/llvm/pull/61

brson · 2017-06-02T18:34:58.391Z

It seems pretty hard to tell whether these are necessary after an upgrade a-priori.

eholk · 2017-06-02T19:45:50.266Z

That’s a smaller set of changes than I was expecting. Just going by the titles, it seems like most of the changes could probably be upstreamed.

eholk · 2017-06-02T20:08:44.597Z

I went ahead and opened https://github.com/rust-lang/rust/issues/42389, since it seems like de-forking LLVM would be beneficial in the long run. Of course, there may still be reasons why the state of affairs is necessary, so the bug seems like a good place to discuss the feasibility and desirability of de-forking LLVM.

cuviper · 2017-06-02T20:20:25.090Z

FWIW, Rust does work with vanilla LLVM. We use the system LLVM in Fedora, and probably most other distros do too. But we’re only trying to support our own native linux-gnu targets in that case.

I do apply some of the rust-llvm backported fixes to our system LLVM though.

vadimcn · 2017-06-02T20:57:12.602Z

This is the big one: https://github.com/rust-lang/llvm/commit/c9bf6e3f98557937df93b24c48b2564c6963e788

As far as I know, LLVM devs don’t want to merge any of that, ever.

system · 2019-03-25T08:28:35.411Z

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.