Please pardon me for possibly being rude, but I’m working on getting ponyc to use LLVM as a submodule instead using whatever is installed on a system when building the source.
I do notice that in actuality rust-lang/llvm is fairly close to llvm-mirror/llvm release_60. I diff’d them and there are only 120 files that are “different” out about 28,000. But the history is quite different, has this been a problem?
Finally, any recommendations of things to or not to do?
Historically, Rust has wanted to land patches that are Rust-specific (e.g., to teach LLVM about our allocation functions). To support this we maintain our own fork of LLVM intended solely for use inside the compiler; I would expect that ponyc would have similar needs eventually.
I don’t know if we have a list of commits and reasoning behind each somewhere – @alexcrichton may know, though.
GitHub won’t let you compare to llvm-mirror, since there’s no “fork” relationship in their view, but you can easily compare commits locally. For example, if you have a clone of rust-lang/llvm, then git remote add llvm https://github.com/llvm-mirror/llvm.git, git remote update, then try commands like:
$ git log --oneline llvm/release_60..
9ad4b7e8d7d1 (HEAD) Merge pull request #117 from alexcrichton/wasm-fix
1e0cfd69bbb7 [WebAssembly] Fix fast-isel lowering illegal argument and return types.
56c931901cfb Merge pull request #116 from alexcrichton/wasm-fix
46c9cc303763 [WebAssembly] Fix the opcode number for i64.load16_u.
ff78b27cae9a Merge pull request #115 from nikic/simplifycfg-latch
823320b6778f Re-apply Relax restriction for folding unconditional branches
1abfd0e562cc Merge pull request #114 from nox/mergefunc
efbf5cc908e1 [MergeFunctions] Fix merging of small weak functions
da0cb60cd6ff Merge pull request #111 from nox/mergefunc
a0ffe4014b5c [PR37339] Fix assertion in FunctionComparator::cmpInlineAsm
fd7dd99edf37 Merge pull request #113 from cuviper/merge-release_60
2de9cac9bdfc Merge branch 'release_60' into rust-llvm-release-6-0-0
efe8743abe16 Merge pull request #112 from alexcrichton/wasm-fix
b6c1a03fb498 [WebAssembly] Teach fast-isel to gracefully recover from illegal return types.
599282dcd587 Merge remote-tracking branch 'origin/rust-llvm-release-6-0-0' into rust-llvm-release-6-0-0
7b308e4d7b0a Work around PATH_MAX not defined
e2a59042654c Work around PATH_MAX not defined
7243155b1c3d (origin/rust-llvm-release-6-0-0-tmp) The signatures of __rust_alloc and __rust_dealloc have changed
6ceaaa4b0176 Merge pull request #109 from varkor/bss-undefined-globals
6baafaafae43 Place undefined globals in .bss instead of .data
0903c72cbbc3 Merge pull request #108 from alexcrichton/fix
aa5b816d0ce8 [DebugInfo] Discard invalid DBG_VALUE instructions in LiveDebugVariables
5f6a0da60e27 Merge remote-tracking branch 'llvm-mirror/release_60' into rust-llvm-release-6-0-0
ba2edd794c7d Merge pull request #107 from bdrewery/llvm-fix-libexecinfo-freebsd
05b5e35bd2f3 Merging r326358: Fix libexecinfo detection on FreeBSD and NetBSD. ------------------------------------------------------------------------ r326358 | dim | 2018-02-28 12:04:21 -0800 (Wed, 28 Feb 2018) | 29 lines Changed paths: M /llvm/trunk/lib/Support/CMakeLists.txt
5c6b7fea4a30 Merge pull request #106 from dotdash/jt_assume
3dd7de023be4 [JumpThreading] PR36133 enable/disable DominatorTree for LVI analysis
90f447e9556a [JumpThreading] Preservation of DT and LVI across the pass
4d4f5d077d67 Merge pull request #105 from glaubitz/rust-llvm-release-6-0-0
24a2acf8cf90 [Sparc] Include __tls_get_addr in symbol table for TLS calls to it
ba52c6ec697b Merge pull request #103 from dotdash/mlsm
43343c49cb72 Mark MergedLoadStoreMotion as not preserving MemDep results
9f81beaf3260 Apply fix from bug 36184
36c8f319d57e [SelectionDAG]: Ignore "returned" in the presence of an implicit sret.
f8c3edcfe062 Partially revert r308329 to fix MSVC
93250244ee5d Disable checks for libatomic for now
74cccb203d72 Fix compile on dist-i686-linux builder
dbcd44f081b4 Add knowledge of __rust_{alloc,realloc,dealloc}
288e4ddb85ea Add accessors for MCSubtargetInfo CPU and Feature tables
There are also commits on release_60 for the imminent 6.0.1 that Rust doesn’t have yet. I’ll submit a PR for this when it’s released, if nobody beats me to it.
I think that llvm-mirror/llvm didn’t actually exist when we first created out fork, but nowadays we definitely follow it! Currently we follow LLVM releases (via llvm-mirror/llvm) by taking their release branches in git and starting from there.
After we branch we add some rust specific patches that are too unstable to land upstream as well as some weird build patches that also don’t make sense to upstream. After that we only take backports of already landed patches upstream.
I think using a fork is a great idea as it gives you some more agility to the exact specifics of what youre doing (aka build shenanigans or Pony-specific things you’d like to test out). @cuviper is right though in that you’ll want to maintain compatibility with stock LLVM releases (aka most of the test suite should pass) for distros and such.
Otherwise the main thing I’d recommend is to not stray too far from upstream LLVM. Projects like PNaCl (IIRC) and Emscripten both strayed really far and it ends up never going upstream and becoming obsoleted in the long run. Patches we used to carry ourselves all bitrotted eventually and made upgrading too difficult so we dropped or merged them upstream.
And finally, I’d prepare to invest heavily in sccache/ccache caching on CI! Performing a full build of LLVM on a 4-core machine on Travis takes ~10m with sccache (caching via S3), and it’s so much easier to deal with CI if you always build LLVM instead of trying to cache it in its entirety.
Have you considered rather than using the current fork from luqmana/llvm creating a new fork from llvm-mirror/llvm, that way the history would be more obvious, I think?
I think using a fork is a great idea as it gives you some more agility to
the exact specifics of what youre doing (aka build shenanigans or
Pony-specific things you’d like to test out). @cuviperhttps://internals.rust-lang.org/u/cuviper is right though in that
you’ll want to maintain compatibility with stock LLVM releases (aka most of
the test suite should pass) for distros and such.
Otherwise the main thing I’d recommend is to not stray too far from
upstream LLVM. Projects like PNaCl (IIRC) and Emscripten both strayed
really far and it ends up never going upstream and becoming obsoleted in
the long run. Patches we used to carry ourselves all bitrotted eventually
and made upgrading too difficult so we dropped or merged them upstream.
Definitely good advice
And finally, I’d prepare to invest heavily in sccache/ccache caching on
CI! Performing a full build of LLVM on a 4-core machine on Travis takes
~10m with sccache (caching via S3), and it’s so much easier to deal
with CI if you always build LLVM instead of trying to cache it in its
entirety.
Ah for this there's so far always been a way to "work without the patch". We've got a #define which says "I'm using the Rust-specific LLVM" and that allows the C/C++ code to operate differently if necessary (the glue we have or the runtime). Otherwise the patches are typically fixing optimization passes or codegen in LLVM itself and don't require any changes at the IR emission layer to make use of them
I'd like to! I just don't trust myself to delete and recreate the repo with full fidelity!
I beleive you can fork llvm-mirror/llvm since your current llvm is not based on llvm-mirror. But you'll probably have to give it a different name or rename your current "llvm" repo.
