# Installing docs is slow on Windows **URL:** \ **Category:** tools and infrastructure\ **Created:** [November 28, 2018, 11:51am UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917 "2018-11-28T11:51:24Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![johnthagen](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/johnthagen/32/2005_2.png) [@johnthagen](https://internals.rust-lang.org/u/johnthagen)\ **Post date:** [November 28, 2018, 11:51am UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/1 "2018-11-28T11:51:24Z")
It’s not a bug in Rust 2018 per se, but running [`rustup update` on Windows is very slow because of an issue installing `rust-docs`](https://github.com/rust-lang/rustup.rs/issues/1540). It hurts the user experience, and I’d hate if users were turned off to Rust because of this paper cut. ---
**Author:** ![DrizztVD](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/drizztvd/32/4521_2.png) [@DrizztVD](https://internals.rust-lang.org/u/DrizztVD)\ **Post date:** [November 28, 2018, 3:14pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/2 "2018-11-28T15:14:26Z")
+1 ---
**Author:** ![Uther](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/uther/32/6225_2.png) [@Uther](https://internals.rust-lang.org/u/Uther)\ **Post date:** [November 28, 2018, 5:53pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/3 "2018-11-28T17:53:29Z")
Yes, it’s an old problem : [https://github.com/rust-lang/rustup.rs/issues/763#issuecomment-437283173](https://github.com/rust-lang/rustup.rs/issues/763#issuecomment-437283173) Rustup is slow to install docs with some anti-viruses. With Trend Micro, it’s actually worse than with Windows defender : the installation can take hours. ---
**Author:** ![DrizztVD](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/drizztvd/32/4521_2.png) [@DrizztVD](https://internals.rust-lang.org/u/DrizztVD)\ **Post date:** [November 28, 2018, 6:09pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/4 "2018-11-28T18:09:07Z")
> [@Announcing Rust 2018 Beta release!](https://internals.rust-lang.org/t/announcing-rust-2018-beta-release/8901/13): > > Rustup is slow to install docs with some anti-viruses. Windows install is still much slower than Linux with zero anti-virus interference. The experience with anti-virus is a deal-breaker. Imagine pitching Rust to a manager and he/she runs into this problem on a company computer which has mandatory endpoint protection. ---
**Author:** ![eddyb](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/eddyb/32/8411_2.png) [@eddyb](https://internals.rust-lang.org/u/eddyb)\ **Post date:** [November 28, 2018, 8:02pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/5 "2018-11-28T20:02:14Z")
I think we should maybe disable docs by default - you generally don’t need them if you’re already using [doc.rust-lang.org](http://doc.rust-lang.org). IIRC part of the problem is NTFS having trouble with many (but small) files and rustdoc creating a lot of them. ---
**Author:** ![steveklabnik](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/steveklabnik/32/4524_2.png) [@steveklabnik](https://internals.rust-lang.org/u/steveklabnik)\ **Post date:** [November 28, 2018, 8:18pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/6 "2018-11-28T20:18:12Z")
Last time we did that, users were upset, so we re-enabled it. People really like offline docs by default, as usually, you don’t _remember_ that you need them before you actually need them. ---
**Author:** ![eddyb](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/eddyb/32/8411_2.png) [@eddyb](https://internals.rust-lang.org/u/eddyb)\ **Post date:** [November 28, 2018, 8:33pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/7 "2018-11-28T20:33:37Z")
Ahh, “offline” is key here. I guess I’ll be eating my words next time I’m on a flight, heh . ---
**Author:** ![steveklabnik](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/steveklabnik/32/4524_2.png) [@steveklabnik](https://internals.rust-lang.org/u/steveklabnik)\ **Post date:** [November 28, 2018, 8:40pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/8 "2018-11-28T20:40:46Z")
> [@Announcing Rust 2018 Beta release!](https://internals.rust-lang.org/t/announcing-rust-2018-beta-release/8901/15): > > IIRC part of the problem is NTFS having trouble with many (but small) files and rustdoc creating a lot of them. As it so happens, I was working on getting some numbers for this, and installing the `rust-docs` component for 1.30.1 copies **15,426** files. So uh, yeah. ---
**Author:** ![mark-i-m](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mark-i-m/32/3167_2.png) [@mark-i-m](https://internals.rust-lang.org/u/mark-i-m)\ **Post date:** [November 28, 2018, 9:20pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/9 "2018-11-28T21:20:07Z")
Can we ship the docs as a zip or a tar by default for Windows only? That way, it would be one big file. When the user tries to use the docs, it could prompt and ask if you want to unzip, with a warning that it might take a while on some file systems. ---
**Author:** ![Uther](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/uther/32/6225_2.png) [@Uther](https://internals.rust-lang.org/u/Uther)\ **Post date:** [November 29, 2018, 5:04am UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/10 "2018-11-29T05:04:38Z")
I don’t believe it is the main part of the problem. Tools like 7-zip can uncompress the docs in a reasonable amount of time. ---
**Author:** ![DanielKeep](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/danielkeep/32/1332_2.png) [@DanielKeep](https://internals.rust-lang.org/u/DanielKeep)\ **Post date:** [November 29, 2018, 5:16am UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/11 "2018-11-29T05:16:40Z")
Potential over-engineered solution: when installing `rustup`, install a minimal HTTP server for hosting Rust docs. It should _just_ serve static files pulled out of compressed archives. Configure it to run on startup, and only load archives if the user actually requests docs from them. An _even better_ solution would be to have a _standardised_ archive format for web pages, but that’s been true for the last two-ish decades and none of the ones created ever got any traction. The closest I’ve seen was CHM, but even Microsoft abandoned that. ---
**Author:** ![jethrogb](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jethrogb](https://internals.rust-lang.org/u/jethrogb)\ **Post date:** [November 29, 2018, 8:43am UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/12 "2018-11-29T08:43:14Z")
> [@Uther](#): > > Tools like 7-zip can uncompress the docs in a reasonable amount of time. Windows is just very very slow when dealing with lots of files. Video game developers have known this for decades which is why they always put game assets in some kind of archive. I couldn't get convincing evidence of 7-zip being much faster. Here are my tests: ### Objective To measure the time it takes to install the rust-docs component with various tools. ### Method The following 4 tests were performed. #### rustup The following procedure is performed on a Windows installation with an out of date `stable` toolchain previously installed with rustup. Process Monitor is installed. It is configured with the following filter: "Include if Process Name contains rustup". It is configured with "Drop filtered events". "Capture events" is enabled. The following command is executed in the command prompt: `rustup update stable`. After completion, "Capture events" is disabled. The event log in Process Monitor is manually checked for the first and last event that have to do with the extraction of the rust-docs component. The data point is the time difference between those two events. #### 7-zip The following procedure is performed on Windows. 7-zip is installed from [https://www.7-zip.org/](https://www.7-zip.org/) . [https://static.rust-lang.org/dist/rust-docs-1.30.0-x86\_64-pc-windows-msvc.tar.xz](https://static.rust-lang.org/dist/rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz) is downloaded. The following batch script is executed: ```rust echo %time% 7z x -so rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz | 7z x -ttar -si echo %time% ``` The data point is the time difference between the two printed times. #### FastCopy The following procedure is performed on Windows. FastCopy is installed from [FastCopy](https://fastcopy.jp/en/) . The files previously extracted with 7-zip are copied to a sibling directory. The data point is the “Total Time” reported by FastCopy. #### tar The following procedure is performed on Linux. [https://static.rust-lang.org/dist/rust-docs-1.30.0-x86\_64-pc-windows-msvc.tar.xz](https://static.rust-lang.org/dist/rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz) is downloaded. The following command is executed: ```rust sync; time (tar xf rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz; sync) ``` The data point is the “real” time reported by the `time` command. ### Setup These tests were performed on the following system: - Lenovo ThinkPad T470s - Toshiba 500GB SSD - Intel Core i7-7600U - 20GB RAM - OS 1 - Windows 10 - Filesystem: NTFS - AV: Windows Defender - OS 2 - Ubuntu 16.04 - Filesystem: ext4 ### Results - rustup (Windows) 3m43s - 7-zip (Windows) 2m35s - FastCopy (Windows) 2m53s - tar (Linux) \<1½s ### Conclusion Rustup on Windows is not significantly slower than other Windows tools such as 7-zip and "the Fastest Copy/Backup Software on Windows" FastCopy. All Windows methods are at least 100× slower than the Linux method. > [@DanielKeep](#): > > Potential over-engineered solution: when installing `rustup` , install a minimal HTTP server for hosting Rust docs. It should _just_ serve static files pulled out of compressed archives. Configure it to run on startup, and only load archives if the user actually requests docs from them. I actually like this idea. It would also fix the issues with the WebFonts not loading properly when viewed from a `file://` URL. Note that `tar` is not an appropriate archive format since it doesn't have an index. ---
**Author:** ![johnthagen](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/johnthagen/32/2005_2.png) [@johnthagen](https://internals.rust-lang.org/u/johnthagen)\ **Post date:** [November 29, 2018, 12:21pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/13 "2018-11-29T12:21:59Z")
@nrc [appears to be working on rustup profiles to help address this issue](https://github.com/rust-lang/rustup.rs/issues/998#issuecomment-432831155). ---
**Author:** ![jethrogb](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jethrogb](https://internals.rust-lang.org/u/jethrogb)\ **Post date:** [November 29, 2018, 12:23pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/14 "2018-11-29T12:23:32Z")
> [@johnthagen](#): > > @nrc [appears to be working on rustup profiles to help address this issue](https://github.com/rust-lang/rustup.rs/issues/998#issuecomment-432831155). No, that's just sidesteps this issue by not installing the docs. It does nothing to improve the speed of docs installation. ---
**Author:** ![leonardo](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/leonardo/32/1344_2.png) [@leonardo](https://internals.rust-lang.org/u/leonardo)\ **Post date:** [November 29, 2018, 12:35pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/15 "2018-11-29T12:35:35Z")
> [@jethrogb](#): > > No, that’s just sidesteps this issue by not installing the docs. It does nothing to improve the speed of docs installation. Is this a Windows issue or a rustup issue? Or some intermediate case? ---
**Author:** ![jethrogb](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jethrogb](https://internals.rust-lang.org/u/jethrogb)\ **Post date:** [November 29, 2018, 12:41pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/16 "2018-11-29T12:41:21Z")
It’s a [Windows issue](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/12?u=jethrogb) ---
**Author:** ![Diggsey](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/diggsey/32/6311_2.png) [@Diggsey](https://internals.rust-lang.org/u/Diggsey)\ **Post date:** [November 29, 2018, 12:53pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/17 "2018-11-29T12:53:18Z")
It might be possible to speed up by parallelising the copy process, as I believe some of the overhead is thread-specific. Another option would be to implement some of the navigation in javascript, so that you have a few large files instead. The docs are pretty bad without javascript enabled to begin with so… might not be a bad idea. ---
**Author:** ![kornel](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kornel/32/2711_2.png) [@kornel](https://internals.rust-lang.org/u/kornel)\ **Post date:** [November 29, 2018, 12:53pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/18 "2018-11-29T12:53:22Z")
Out of 14639 HTML files, 6450 are `.?.html` which seem to be redirects (tiny files with “redirecting to \”). Maybe these could be removed, and whatever other process needs them (search? cross-linking?) updated to generate final destination links in the first place. ---
**Author:** ![DanielKeep](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/danielkeep/32/1332_2.png) [@DanielKeep](https://internals.rust-lang.org/u/DanielKeep)\ **Post date:** [November 29, 2018, 12:54pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/19 "2018-11-29T12:54:51Z")
I _believe_ they are there because the naming scheme has changed in the past. Those redirects exist to ensure that when `rustdoc` is used to generate online docs, old links continue to work. As such, they could _probably_ be omitted for local copies. ---
**Author:** ![kornel](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kornel/32/2711_2.png) [@kornel](https://internals.rust-lang.org/u/kornel)\ **Post date:** [November 29, 2018, 12:55pm UTC](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917/20 "2018-11-29T12:55:34Z")
Slow installation of docs is a problem because user has to wait. How about launching docs install as a separate background installation process? 1. Install Rust and libs 2. Launch docs install as async 3. Announce Rust is installed 4. (invisible to user) docs will finish a few minutes later [Next page](https://internals.rust-lang.org/t/installing-docs-is-slow-on-windows/8917.md?page=2)