# Rust 2019: Think Bigger **URL:** \ **Category:** Uncategorized\ **Created:** [December 11, 2018, 11:03pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003 "2018-12-11T23:03:10Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![fitzgen](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/fitzgen/32/3625_2.png) [@fitzgen](https://internals.rust-lang.org/u/fitzgen)\ **Post date:** [December 11, 2018, 11:03pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/1 "2018-12-11T23:03:10Z")
[http://fitzgeraldnick.com/2018/12/11/rust-2019-think-bigger.html](http://fitzgeraldnick.com/2018/12/11/rust-2019-think-bigger.html) ---
**Author:** ![withoutboats](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/withoutboats/32/4560_2.png) [@withoutboats](https://internals.rust-lang.org/u/withoutboats)\ **Post date:** [December 12, 2018, 2:16pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/2 "2018-12-12T14:16:20Z")
I like everything in this post a lot. I especially think its insightful that precompiled artefacts through [crates.io](http://crates.io) is probably going to provide the biggest compile time win on average, rather than improving compiler performance directly. It’s something we’ve thought about, but I haven’t seen its relative impact to other things we’re working more seriously on pointed out in this way before. Its a complicated project though. ---
**Author:** ![matklad](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/matklad/32/12266_2.png) [@matklad](https://internals.rust-lang.org/u/matklad)\ **Post date:** [December 12, 2018, 3:16pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/3 "2018-12-12T15:16:14Z")
> [@withoutboats](#): > > I especially think its insightful that precompiled artefacts through [crates.io](http://crates.io) is probably going to provide the biggest compile time win on average, rather than improving compiler performance directly Can we actually measure this? I fear that the win wouldn't be as big as some might hope, due to the fact that most time is spend on codegen and that a significant amount of code-gen happens in the leaf crate due to monomorphisation? Note that precompiled artifacts won't affect the edit-build-run _cycle_, only the first iteration of it. That said, I've recently [experimented](https://github.com/rust-lang/cargo/issues/5885#issuecomment-444262200) with adding caching of `target/` for crates-io dependencies on CI, and wins were substantial. ---
**Author:** ![steveklabnik](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/steveklabnik/32/4524_2.png) [@steveklabnik](https://internals.rust-lang.org/u/steveklabnik)\ **Post date:** [December 12, 2018, 3:27pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/4 "2018-12-12T15:27:17Z")
> [@withoutboats](#): > > is probably going to provide the biggest compile time win on average Doesn't this only help the first compile, though? That's not generally the largest pain point, in my experience. ---
**Author:** ![withoutboats](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/withoutboats/32/4560_2.png) [@withoutboats](https://internals.rust-lang.org/u/withoutboats)\ **Post date:** [December 12, 2018, 3:31pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/5 "2018-12-12T15:31:05Z")
> [@steveklabnik](#): > > Doesn’t this only help the first compile, though? That’s not generally the largest pain point, in my experience. Or any time you add a dependency or update your compiler version. But @matklad is probably right that we're being really impacted by monomorphization. So much of that library code is being compiled again and again because its monomorphized into the downstream crate. ---
**Author:** ![petrochenkov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/petrochenkov/32/2784_2.png) [@petrochenkov](https://internals.rust-lang.org/u/petrochenkov)\ **Post date:** [December 12, 2018, 3:45pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/6 "2018-12-12T15:45:39Z")
> Or any time you add a dependency or update your compiler version. Or any time you do `cargo install` or `cargo install-update` for some tool pulling in half of [crates.io](http://crates.io) to build, like `racer` or `cargo-vendor`. ---
**Author:** ![matklad](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/matklad/32/12266_2.png) [@matklad](https://internals.rust-lang.org/u/matklad)\ **Post date:** [December 12, 2018, 3:51pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/7 "2018-12-12T15:51:21Z")
Note also this thread: [[Idea] Cargo Global Binary Cache](https://internals.rust-lang.org/t/idea-cargo-global-binary-cache/9002) I think it’ll get us 70% of shared [crates.io](http://crates.io) cache with a small amount of implementation & design work. Fun observation: there’s a desire to add “rlib dependencies” feature to Cargo, to integrate with other build systems and such. This is a huge design space. However, just sharing `target` dir for all `crates.io` work will get us shared cache aspect of rlib dependencies, and is very simple because the layout of target dir is controlled by Cargo and is an implementation detail. Basically, we need to tweak a couple of lines in Cargo to say "if this dep comes from [crates.io](http://crates.io), compile it to `~/.cargo/target`" and not to `./target`. ---
**Author:** ![withoutboats](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/withoutboats/32/4560_2.png) [@withoutboats](https://internals.rust-lang.org/u/withoutboats)\ **Post date:** [December 12, 2018, 3:55pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/8 "2018-12-12T15:55:08Z")
Let’s avoid getting too deep in the weeds on this thread. Good thing for the cargo team to discuss! ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [December 12, 2018, 6:03pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/9 "2018-12-12T18:03:53Z")
While I appreciate that a binary cache would speed up cargo, I’d _really_ like to avoid having one enabled by default, and having people get used to the idea of downloading and running binaries. ---
**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\ **Post date:** [December 12, 2018, 6:24pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/10 "2018-12-12T18:24:10Z")
For this specifically, you’re already downloading code, compiling it to a binary, and running that binary. Downloading and running a binary is no more problematic, as you still have to trust the distribution, _unless_ you’re in the habit of auditing the actual code you’ve downloaded. Said cache would definitely want to be the one providing the compiled binaries rather than trusting uploaders to send the correct one, so that you only have to trust one source instead of many, but you still have to trust the distributer. I don’t see how downloading a binary is any more dangerous than downloading the code and compiling it (other than local CPU time). If the host is complicated (knock on wood), they can alter either. If hashes are used to prevent this from a separate source, they prevent either, or if from the same source, neither. Yes, downloading unsandboxed code and running it is a security risk, and it’s one that’s only solved by trusting the source. But the risks are the same for pre-compiled or not. You can even stick in a verification that the pre-compiled binary is the same as what you get building the code when you audit it, if you’re actually auditing the code. If it’s been built already, a trusted source confirms it’s built correctly, what would you benefit from building it again? (This is assuming it’s being built with the same flags of course, if you have a rare configuration you’d just build it locally.) TL;DR: assuming reproducible builds, what risk does downloading a hash-verified binary have that a hash-verified source doesn’t? As far as I can tell, the trust problem is identical to both of them. ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [December 12, 2018, 6:31pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/11 "2018-12-12T18:31:58Z")
You’re assuming reproducible builds and someone auditing the binaries to make sure that they were built appropriately. Are we talking about having a central service build all Rust code once? And if so, how do we make sure _that_ service (a high-value target) doesn’t get compromised? (Especially when it builds random Rust crates.) That said, I can think of one advantage to building all Rust crates once. Imagine if, when a crate gets uploaded with `cargo publish`, it goes into a staging area until built, and if it doesn’t build (in an isolated environment with only the declared dependencies and no network access) then it gets rejected. ---
**Author:** ![notriddle](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/notriddle/32/14082_2.png) [@notriddle](https://internals.rust-lang.org/u/notriddle)\ **Post date:** [December 12, 2018, 11:44pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/12 "2018-12-12T23:44:57Z")
> [@josh](#): > > if it doesn’t build (in an isolated environment with only the declared dependencies and no network access) then it gets rejected But there are existing crates that have dependencies other than the declared ones. Like `openssl-sys`, which additionally depends on OpenSSL, and `winapi`, that depends on a bunch of closed-source DLLs. And those crates are useful, and you can't legally bundle the source code for kernel32.dll. ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [December 14, 2018, 7:58pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/13 "2018-12-14T19:58:53Z")
winapi is quite capable of building without access to those proprietary DLLs. And we need better solutions to handle dependencies on third-party libraries. ---
**Author:** ![gus](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/gus/32/2924_2.png) [@gus](https://internals.rust-lang.org/u/gus)\ **Post date:** [December 18, 2018, 5:37am UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/14 "2018-12-18T05:37:17Z")
Fyi: Some popular source licenses don’t allow redistribution in binary form, or have additional obligations in this case. ---
**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\ **Post date:** [December 18, 2018, 4:20pm UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/15 "2018-12-18T16:20:09Z")
> [@gus](#): > > Some popular source licenses don’t allow redistribution in binary form That wouldn't be an Open Source license. > [@gus](#): > > or have additional obligations in this case. Primarily distributing corresponding source code, which any service hosting binaries of crates would have to do. ---
**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\ **Post date:** [March 25, 2019, 8:31am UTC](https://internals.rust-lang.org/t/rust-2019-think-bigger/9003/16 "2019-03-25T08:31:17Z")
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.