Installing docs is slow on Windows

[johnthagen]

@nrc appears to be working on rustup profiles to help address this issue.

[jethrogb]

@nrc appears to be working on rustup profiles to help address this issue.

No, that’s just sidesteps this issue by not installing the docs. It does nothing to improve the speed of docs installation.

[leonardo]

No, that’s just sidesteps this issue by not installing the docs. It does nothing to improve the speed of docs installation.

Is this a Windows issue or a rustup issue? Or some intermediate case?

[jethrogb]

It’s a Windows issue

[Diggsey]

It might be possible to speed up by parallelising the copy process, as I believe some of the overhead is thread-specific.

Another option would be to implement some of the navigation in javascript, so that you have a few large files instead. The docs are pretty bad without javascript enabled to begin with so… might not be a bad idea.

[kornel]

Out of 14639 HTML files, 6450 are .?.html which seem to be redirects (tiny files with “redirecting to <link>”). Maybe these could be removed, and whatever other process needs them (search? cross-linking?) updated to generate final destination links in the first place.

[DanielKeep]

I believe they are there because the naming scheme has changed in the past. Those redirects exist to ensure that when rustdoc is used to generate online docs, old links continue to work.

As such, they could probably be omitted for local copies.

[kornel]

Slow installation of docs is a problem because user has to wait. How about launching docs install as a separate background installation process?

1. Install Rust and libs
2. Launch docs install as async
3. Announce Rust is installed
4. (invisible to user) docs will finish a few minutes later

[DanielKeep]

invisible to user

As someone using spinning rust, having a background process hammering the disk is absolutely visible to the user. I’d be angry at an installer doing that in the background where I can’t stop it, without any indication as to how long it’ll be before I can use the disk again.

[Uther]

Too me 3-4 minutes is bad, but not a unbearable amount of time. To feel reassured, a progress indicator would be enough.

But with Trend Micro Office Scan, you really wonder if rustup is not hanging : it may take hours to install the docs.

[passcod]

Unsure how that would affect the experience, but Windows does famously have their CHM format, which is essentially (I think) a zip of many HTML files, that are directly viewable natively without extracting. Could this be used?

[DanielKeep]

[…] Windows does famously have their CHM format, […]

Which (AFAIK) is deprecated, and doesn’t support Unicode. Also, the HTML Help viewer itself is pretty janky, what with it effectively being an embedded IE with a table of contents (good), and no tabs (very bad).

[DrizztVD]

Of the docs, core has 10 123 files and std 2722. Removing all the files with little content in them is one workaround. Adding them together into a larger file is another. A script could do so, perhaps.

Windows Search file indexer is part of the problem.

Creating a PDF from the web pages could be possible: https://helpx.adobe.com/acrobat/using/converting-web-pages-pdf.html

[johnthagen]

To feel reassured, a progress indicator would be enough.

I created an issue to consider creating a progress bar for rustup component installs. While this in no way solves the problem, I think it is at least a step forward and (if not too difficult to implement) is universally helpful.

[bluss]

There are a lot of extra files that redirect, like from_mut.v.html is a redirect to fn.from_mut.html and so on for every function and type, at least. Is there a plan to move away from these? It would be good to find some point where they can be removed or replaced with some built in rustdoc smarts.

[Kroc]

The docs directory can take up a lot of space too. If NTFS compression was enabled, it would take up a lot less space and less blocks would have to be actually written to disk, ultimately saving time.

[RReverser]

I’m lately mostly using devdocs.io which just runs in the browser and allows precaching docs for various languages and libs (including Rust ones) for offline use by storing them in the IndexedDB. Could regular Rust docs switch to the same approach? Then we wouldn’t even need an installation step nor a problematic number of files. All that user would have to do is open https://doc.rust-lang.org/ to have docs cache updated to the latest one.

[DrizztVD]

I’m lately mostly using devdocs.io which just runs in the browser and allows precaching docs for various languages and libs

That looks like an almost-good approach. It looks like devdocs is missing some of the docs though. Also, the caching is not 100% reliable it seems. this could be fixed, perhaps. It’s similar to the http server approach in some ways.

[kornel]

My initial thought was that’s it’s super tricky to make rustup “install” the offline docs in the browser and ensure they stay long enough… but that’s actually not necessary. We can assume Rust users will open the docs page anyway, and they’ll probably use it often enough to keep it cached.

So I like that. It entirely shifts the problem from “docs are slow to install” to “docs don’t even need an install”.

[DanielKeep]

The whole point of installing docs locally is to… install them locally. For when you don’t have a network connection, or for computers that don’t/aren’t allowed to have unfettered internet access.

Relying on an unaffiliated, third-party web service that only serves the current stable docs (which might or might not be what you’re using), in an unfamiliar structure, and which has an apparently unreliable caching system would seem to defeat the purpose.