Need help with emscripten port

[tomaka]

I just found out about this file: https://github.com/kripken/emscripten/blob/42fb486c53d627b203b77af6e5d0c088c0ad03ad/src/library_pthread_stub.js

Which is supposed to be included if you don't use pthreads: https://github.com/kripken/emscripten/blob/42fb486c53d627b203b77af6e5d0c088c0ad03ad/emcc#L1041

I'll continue to investigate to see why that doesn't work. EDIT: The answer is that it's because the missing symbols are pthread_attr_setstack and pthread_rwlock_rdlock, which aren't provided.

[brson]

The only way to enable pthread symbols is to use the USE_PTHREADS option, which is documented here. But as I said above it then tries to use the document object, which doesn't exist in node.

It's probably not worth trying to use pthreads in emscripten until they work in wasm+firefox, at least by default. I think the current implementation uses incredibly expensive continuations.

Maybe we may want to ping kripken on this problem.

He'll be happy to help.

[tomaka]

For the record, now that I cleaned up most of the mess in the logs the main remaining problems are, other than pthreads:

• Exceptions don't work. I'm going to try fixing it, but I'm not familiar with how unwinding works under the hood.
• More worrisome, the tests that call an external C function and get back the values don't work for u8s and u16s (1 2 3 4). They simply get back the value 0.

[brson]

More worrisome, the tests that call an external C function and get back the values don't work for u8s and u16s (1 2 3 4). They simply get back the value 0.

These are struct returns, which have very platform-specific ABIs. The Rust definition of the emscripten C ABI just reuses the arm definition. Emscripten is fairly arm-ish, but seemingly not here.

The fix is probably going to involve giving emscripten it's own C ABI definition and futzing with the classify_ret_ty fn.

[tomaka]

Down to 17 failures: http://pastebin.com/V6R8aszM

• 3 missing Math_trunc. The actual error is that llvm_trunc_f64 is missing ; this error message is due to a fix attempt in my emscripten fork. This should be quite easy to fix however.
• 4 because of wrong ABI with C code.
• 1 because #[repr(packed)] doesn't work correctly.
• 1 wrong u64 alignment (but this is likely the test being wrong with its expected value).
• 1 LLVM ERROR: invalid vector instr with the SIMD test.
• 2 missing _Exit symbols. The tests themselves will need to be ignored since they rely on spawning a child process, but I'd like to understand why this linking error happens.
• 5 tests that must be ignored.

[tomaka]

The #[repr(packed)] issue is probably related to https://github.com/rust-lang/rust/issues/27060 Emscripten just assumes that all u32s and all u64s are 4-bytes aligned. If it isn't the case there isn't even a slow path, instead it just returns the wrong value.

Down to 7 failures right now, but it becomes more and more tedious.

[tomaka]

(hopefully I'm not spamming this topic ; I use it as a kind of "meta-issue for emscripten")

Only 2 failures left in debug mode: the repr(packed) issue and an unsupported SIMD operation. In release mode around twenty failures, and all the new ones are caused by the strings-related problem I talked about above.

But eddyb found out that the string problem was likely caused by a faulty LLVM pass. I'm currently compiling without this line to see if it works: https://github.com/kripken/emscripten-fastcomp/blob/18f61c430b6d26ac0b3da49d087fa9ca21131a2e/lib/Target/JSBackend/JSBackend.cpp#L3762

[tomaka]

For the record the bug was actually due to the SimplifyStructRegSignatures pass, that converted some functions from the ret_type fn() signature to void fn(ret_type* output) but didn't remove the pure attribute from them when doing so.

This led to some important function calls (like CStr::from_ptr) being stripped because LLVM thought they were useless.

[tomaka]

@brson Is there a reason not to merge your changes to librustc_llvm, configure and main.mk into master?

[brson]

@tomaka. Yes. Unfortunately, they depend on LLVM features that only exist in emscripten's port, like adding "js" to LLVM_TARGETS.

There may be some pieces that can be separated out.

[tomaka]

Ah, right (I feel stupid for asking that).

I guess the right thing to do is somehow detect in the build process whether LLVM supports the JS backend?

[brson]

That would be great. I don't know how to do it though.

[bpowers]

llvm-config can do that:

~/PATH_TO_LLVM_BIN/llvm-config --targets-built

will return a space-separated list of targets, like:

X86 JSBackend WebAssembly

I'm not exactly sure how to get that into the rust build process :\

[brson]

Some updates. I'm hoping to push emscripten support across the finish line in the next 2 or so months. That is, for the project to distribute emscripten-targetting compilers and stds (whether they contain awful bugs is another matter).

The short term plan is for kripken to port emscripten's LLVM branch forward to the same merge-base as ours, then for us to rebase onto their branch. We don't have any immediate plans to upgrade LLVM, but in the future we'll have try not to do it too often, because each time will force emscripten to upgrade as well. Likewise we'll have to upgrade when they do, but they don't anticipate needing to either. And we'll have to tolerate temporary breakage of our emscripten support if it gets difficult to do it simultaneously. It could end in disaster. Who knows.

Once we're on their LLVM, distributing emscripten bins is mostly a matter of turning some knobs on the CI.

In the longer term, kripken is very keen on creating a MIR->wasm backend for Rust, bypassing LLVM. If it works out then it could open up a lot of really interesting possibilities.

@tomaka is the best way to compile for emscripten still to use my 'emscripten' branch or do you have something more up to date? I'll aim to rebase it onto Rust master soon.

[tomaka]

@tomaka is the best way to compile for emscripten still to use my 'emscripten' branch or do you have something more up to date? I'll aim to rebase it onto Rust master soon.

The Reddit article that was posted a few months ago was slightly more up to date, but not by much.

[eminence]

It could end in disaster. Who knows.

Or it could end in success! I'm super happy you all are trying to make this work. Thanks for all the effort!

[brson]

It turns out that although our LLVM fork is further ahead than emscripten's chronologically, that they are actually ahead of us logically. We are on 3.8 and they are somewhere in 3.9. So we're going to have to upgrade. Unfortunately, this means we have to switch from make to cmake for building llvm, which is going to take some unusual effort. I'm going to put it on my todo list.

[brson]

We're using CMake now to build LLVM. So the path is clear to rebase the Rust and emscripten ports to the same merge-base. The plan is going to be complicated by the upcoming AVR port, which also wants to impose some LLVM churn.

Right now I think the best thing to do is for Rust to port forward to LLVM master (which we're going to need to do no matter what), then coordinate with @shepmaster to figure out how much more upgrading we need to do on AVR, and @kripken to rebase onto the same commit.

Anybody interested in doing the LLVM forward port?

cc @alexcrichton

[jer]

I'd really like to help with making this happen, though I have no deeper knowledge about the inners of rustc or LLVM.

[shepmaster]

Anybody interested in doing the LLVM forward port?

As a potential starting point, I have some hacked up commits that might at least serve as guideposts for where Rust needs to be updated to accommodate LLVM 3.9.