Need help with emscripten port

brson 2016-09-05 23:49:54 UTC #99

@rschulman Yes, that's right. Usually that folder doesn't exist, though perhaps it's created if you run cargo directly in src/bootstrap.

brson 2016-09-06 00:37:40 UTC #100

I've merged @rschulman's fixes to the test suite into my emscripten-new branch. Thanks @rschulman!

It sounds like there are still failing tests in stdtest, like those that spawn threads. These can similarly have cfg_attr(target_os = "emscripten", ignore) attributes. @rschulman is going to look into those as well, but doesn't have a great deal of time this week.

Once --step check-target --target=asmjs-unknown-emscripten passes (with lots of ignored test cases if necessary) we can get PRs into rust-lang/rust. I'd like to do that by the end of this week, though friday/saturday is rustconf so it's going to be difficult.

jer 2016-09-06 09:20:16 UTC #101

So I gave a talk yesterday at the Rust Cologne Meetup. Slides are online and I have some examples online as well.

With that out of the way, I now hopefully find some time to work on the remaining issues as well.

eminence 2016-09-06 13:21:55 UTC #102

I've wanted to play with this, but have been struggling to get rust built. Thank you so much for providing the docker image!

brson 2016-09-06 21:17:31 UTC #103

I've pushed several commits to my emscripten-new branch. These improve testing support and add a wasm32-unknown-emscripten target that is sufficient to run wasm hello.rs in node using the binaryen interpreter. This branch now requires a commit of libc from my fork as well as my fork of llvm.

I'm going to go ahead and submit PRs for all this stuff now and try to get it landed, while I continue to get the test suite to run cleanly for asmjs.

brson 2016-09-06 21:37:44 UTC #104

I've posted patches to libc and llvm.

brson 2016-09-07 04:47:04 UTC #105

I've opened another issue on the issue tracker to track progress of landing this work. The issue I was previously using is about mir2wasm, and I accidentally commandeered it temporarily to track this work.

brson 2016-09-07 16:59:57 UTC #106

So now my current emscripten-new branch seems to cleanly run the tests for asmjs-unknown-emscripten. The libc wasm32 patches are upstream. My next steps are to get the fastcomp patches merged, rebase brson/emscripten-new onto rust-lang/master, and post a PR to Rust.

In the meantime @rschulman is looking to get compiletest working with wasm32-unknown-emscripten. There's a PR against emscripten to add some missing math intrinsics.

Additional work for people to pick up include:

Going through all the many failing asmjs-unknown-emscripten tests and seeing if any can be fixed. A lot of them seem like they relate to various options that may be configurable in emscripten (like resizable heaps, networking and filesystem support), and it's not clear how or whether we should deal with them in-tree (I suspect they can be dealt-with as -C link-args and std doesn't need to care, but maybe the test runner should be tweaked to change the emcc configuration. not sure; needs investigation).
Fixing unwinding. This is a deep problem and requires fixes in Rust and emscripten. Needs a hero.
Start coming up with mindblowing demos and libraries. An idea I had this morning was a project to collect a few production-ready javascript libraries written in Rust that web programmers can just download and use without dealing with the Rust side. Present it in a nice website that looks appealing to web devs like a real product.

bjorn3 2016-09-07 17:37:43 UTC #107

I would like it if someone writes a opengl/webgl demo.

leeola 2016-09-07 18:33:50 UTC #108

@jer are the Rust source files available for your examples (http://www.hellorust.com/emscripten)? If not, would it be possible to post them?

brson 2016-09-08 02:17:36 UTC #109

Here's the patch to Rust.

jer 2016-09-08 09:54:51 UTC #110

I'll add them, yep

brson 2016-09-14 17:39:05 UTC #111

Update: progress slow on the PR. Right now I'm testing changes to address feedback. Then I'm going to rebase to incorporate the nodejs detection from https://github.com/rust-lang/rust/pull/36256. Then merge. There are two major issues with asmjs/wasm support:

When enabling asmjs/wasm, jemalloc is disabled for all targets. This must be fixed before we can create nightlies. This would be a useful thing for somebody to look into. I don't see myself doing it today.
Unwinding of course is busted. Good thing for somebody to look into. I probably won't be able to prioritize this myself for a long time. A possible approach here is to add an unwinder implementation that targets libcxxabi (which emscripten does support) instead of targeting libunwind directly.

rschulman 2016-09-20 15:35:25 UTC #112

I fell off the grid for the past couple weeks with a huge work event yesterday plus a devastating cold courtesy of my son that I'm only just getting over now. I'm eager to dig back in on this so @brson let me know if there's something specific it might be good for me to look at. In the meantime I'll see if I can learn what jemalloc even does.

aidanhs 2016-09-21 00:03:51 UTC #113

Disclaimer: I'm not very au fait with unwinding, exceptions etc.

Is supporting unwinding actually something we want for rust->asm.js? Exception handling in C++ is pretty bad for asm.js performance and I don't know if 'just' unwinding is any better - you still need to know that you're out of the normal flow of code and impose the overhead/bloat on every function call. What about this suggestion:

i.e. the emscripten target just doesn't support unwinding at all. Unlike C++ (where exceptions can be used as control flow) that's very discouraged in rust, so it feels much more reasonable to make aborting the default and only option?

brson 2016-09-22 19:50:18 UTC #114

@rschulman As we discussed on IRC getting the wasm32 target successfully running tests is a good task. Right now the way compiletest launches node, the emscriptened javascript entry point fails to find the wasm file (https://github.com/kripken/emscripten/issues/4542). The fix for this is to have compiletest change the directory before running the test, but this requires snaking a bunch of arguments through the compiletest codebase and ensuring that all the paths involved in the spawned process still resolve correctly.

Another useful task would be to get the asmjs target working without the big LLVM fastcomp backend (https://github.com/rust-lang/rust/issues/36356). We actually don't need the pnacl legalizer or the asmjs code generator, since we just pass bitcode to emcc which then uses those backends to generate asmjs. But rustc does currently call the LLVM TargetRegistry, passing the asmjs triple, so that LLVM can tell us the right TargetMachine to create. It's not clear that rustc even needs to create a TargetMachine at all for what we are doing. There are two possible solutions here: delete all the LLVM fastcomp code except that which supports the TargetRegistry; make rustc work without instantiating a TargetMachine.

Myself, I'm still poking at the unwinder(https://github.com/rust-lang/rust/issues/36514), and in turn enabling errors on unimplemented symbols (https://github.com/rust-lang/rust/issues/36515). I pushed a patch to the PR that implements unwinding on top of the C++ APIs that emscripten already suppports and it works in cursory testing. I also have an action item to get the automation set up in our dev environment, but haven't started.

What does it mean for exception handling performance to be bad in asm.js? Is it the runtime performance of unwinding and catching exceptions or the bloat associated with the landing pads? Does unwinding impose overhead on the non-unwinding path? I imagine it's similar to the native case; unwinding is just always expensive in code size and run time, but I have seen some decompiled asm.js that looks like there may be extra book-keeping on the non-unwinding path, which would certainly be unattractive.

Unwinding is part of the Rust semantics and we need to support it for the test suite to work in any reasonable fashion if nothing else (otherwise every test would simply abort on failure, which makes for an unpleasant debugging experience when there are multiple test failures). I expect -C panic=abort to work for asmjs targets similarly to others so those that don't want unwinding don't need to pay the cost, and I'd expect most projects that want to deploy to the web to use it for the performance / bloat wins.

brson 2016-09-22 20:02:04 UTC #115

Oh, I haven't mentioned on thread here but the jemalloc problems are fixed in my PR as well.

Other good things to work on: fixing test cases, creating compelling demos.

brson 2016-09-22 22:53:07 UTC #116

I looked into the impact of unwinding on emscripten-compiled code and indeed for both asm.js and wasm32 it is terrible. Every call that may unwind makes a round-trip through a JS try/catch block.

Diggsey 2016-09-27 18:00:01 UTC #117

Other than javascript exceptions, and explicit branching, it should be possible to implement unwinding manually: ie. maintain our own unwind info in a separate stack, and use that to run destructors without actually changing the control flow.

Javascript exceptions would still be used to jump out, but try/catchwould only be necessary whencatch_unwind is called, since those are the only places that execution can resume.

brson 2016-09-27 18:18:59 UTC #118

We could definitely implement a different unwinding scheme on the Rust side. Another variation is return-based unwinding, where every call returns an implicit flag indicating the unwinding path. That would not use the js unwind machinery at all. I've always wanted such an alternate unwinding scheme anyway, though I can't remember the other use-cases offhand... certainly it's useful in places where zero-cost unwinding doesn't exist (like emscripten).