It’s not a bug in Rust 2018 per se, but running rustup update on Windows is very slow because of an issue installing rust-docs. It hurts the user experience, and I’d hate if users were turned off to Rust because of this paper cut.
3 Likes
+1
Yes, it’s an old problem : https://github.com/rust-lang/rustup.rs/issues/763#issuecomment-437283173
Rustup is slow to install docs with some anti-viruses. With Trend Micro, it’s actually worse than with Windows defender : the installation can take hours.
Windows install is still much slower than Linux with zero anti-virus interference. The experience with anti-virus is a deal-breaker. Imagine pitching Rust to a manager and he/she runs into this problem on a company computer which has mandatory endpoint protection.
I think we should maybe disable docs by default - you generally don’t need them if you’re already using doc.rust-lang.org.
IIRC part of the problem is NTFS having trouble with many (but small) files and rustdoc creating a lot of them.
1 Like
Last time we did that, users were upset, so we re-enabled it.
People really like offline docs by default, as usually, you don’t remember that you need them before you actually need them.
5 Likes
Ahh, “offline” is key here. I guess I’ll be eating my words next time I’m on a flight, heh .
2 Likes
As it so happens, I was working on getting some numbers for this, and installing the rust-docs component for 1.30.1 copies 15,426 files.
So uh, yeah.
Can we ship the docs as a zip or a tar by default for Windows only? That way, it would be one big file. When the user tries to use the docs, it could prompt and ask if you want to unzip, with a warning that it might take a while on some file systems.
10 Likes
I don’t believe it is the main part of the problem. Tools like 7-zip can uncompress the docs in a reasonable amount of time.
Potential over-engineered solution: when installing rustup, install a minimal HTTP server for hosting Rust docs. It should just serve static files pulled out of compressed archives. Configure it to run on startup, and only load archives if the user actually requests docs from them.
An even better solution would be to have a standardised archive format for web pages, but that’s been true for the last two-ish decades and none of the ones created ever got any traction. The closest I’ve seen was CHM, but even Microsoft abandoned that.
3 Likes
Windows is just very very slow when dealing with lots of files. Video game developers have known this for decades which is why they always put game assets in some kind of archive.
I couldn't get convincing evidence of 7-zip being much faster. Here are my tests:
Objective
To measure the time it takes to install the rust-docs component with various tools.
Method
The following 4 tests were performed.
rustup
The following procedure is performed on a Windows installation with an out of date stable toolchain previously installed with rustup. Process Monitor is installed. It is configured with the following filter: "Include if Process Name contains rustup". It is configured with "Drop filtered events". "Capture events" is enabled. The following command is executed in the command prompt: rustup update stable. After completion, "Capture events" is disabled. The event log in Process Monitor is manually checked for the first and last event that have to do with the extraction of the rust-docs component. The data point is the time difference between those two events.
7-zip
The following procedure is performed on Windows. 7-zip is installed from https://www.7-zip.org/ . https://static.rust-lang.org/dist/rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz is downloaded. The following batch script is executed:
echo %time%
7z x -so rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz | 7z x -ttar -si
echo %time%
The data point is the time difference between the two printed times.
FastCopy
The following procedure is performed on Windows. FastCopy is installed from FastCopy . The files previously extracted with 7-zip are copied to a sibling directory. The data point is the “Total Time” reported by FastCopy.
tar
The following procedure is performed on Linux. https://static.rust-lang.org/dist/rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz is downloaded. The following command is executed:
sync; time (tar xf rust-docs-1.30.0-x86_64-pc-windows-msvc.tar.xz; sync)
The data point is the “real” time reported by the time command.
Setup
These tests were performed on the following system:
- Lenovo ThinkPad T470s
- Toshiba 500GB SSD
- Intel Core i7-7600U
- 20GB RAM
- OS 1
- Windows 10
- Filesystem: NTFS
- AV: Windows Defender
- OS 2
- Ubuntu 16.04
- Filesystem: ext4
Results
- rustup (Windows) 3m43s
- 7-zip (Windows) 2m35s
- FastCopy (Windows) 2m53s
- tar (Linux) <1½s
Conclusion
Rustup on Windows is not significantly slower than other Windows tools such as 7-zip and "the Fastest Copy/Backup Software on Windows" FastCopy. All Windows methods are at least 100× slower than the Linux method.
I actually like this idea. It would also fix the issues with the WebFonts not loading properly when viewed from a file:// URL. Note that tar is not an appropriate archive format since it doesn't have an index.
8 Likes
No, that's just sidesteps this issue by not installing the docs. It does nothing to improve the speed of docs installation.
Is this a Windows issue or a rustup issue? Or some intermediate case?
It’s a Windows issue
It might be possible to speed up by parallelising the copy process, as I believe some of the overhead is thread-specific.
Another option would be to implement some of the navigation in javascript, so that you have a few large files instead. The docs are pretty bad without javascript enabled to begin with so… might not be a bad idea.
Out of 14639 HTML files, 6450 are .?.html which seem to be redirects (tiny files with “redirecting to <link>”). Maybe these could be removed, and whatever other process needs them (search? cross-linking?) updated to generate final destination links in the first place.
1 Like
I believe they are there because the naming scheme has changed in the past. Those redirects exist to ensure that when rustdoc is used to generate online docs, old links continue to work.
As such, they could probably be omitted for local copies.
1 Like
Slow installation of docs is a problem because user has to wait. How about launching docs install as a separate background installation process?
- Install Rust and libs
- Launch docs install as async
- Announce Rust is installed
- (invisible to user) docs will finish a few minutes later