How do I modify `rustdoc` (the tool run by `cargo doc`)

EDIT: After some discussion, this question is still open, and have its place in this forum.

Note: I am still really new to rust, uncomfortable with the tools used internally by cargo, and the code of rustc in general.

I was trying to implement the RFC that I started to work on. I need to modify how documentation comments are parsed by cargo doc , as well as what is tested by cargo test . I was expecting to have to modify a few lines in rustc , build it with cargo build , then search for the generated binaries and somehow invoke them with cargo doc and cargo test inside a test crate that use the new syntax in the documentation comments that I implemented.

So the first step I did was to the guide to rustc development, especially the chapter on how to build and run.

Witch stage do I need? I only have 43 GB free on my SSD, and I wasn't expecting it to be not enough, but apparently I can't even build the full stage 1. Is stage 0 enough? I think I only need cargo , and rustdoc , so the following should be enough, isn't it?

./x.py build -i --stage 0 src/tools/cargo src/tools/rustdoc

The next step requires to use rustup . I'm on fedora 30, and rustup doesn't seems to be packaged. How do I create my toolchain? And how do I invoke it? Given that I just need to be able to generate the html documentation of a single test file, and rust the test on that file, if it is simpler to just invoke rustc directly, I am totally fine with this. I just don't know what cargo doc nor cargo test do internally.

Do I need to know anything more?

PS - I don't know which tag I should add.

You should post this help query in the Rust users forum. This forum is for discussion of language and compiler enhancements and related issues.

Ah. I was going to post here. Thought I should post in the user forum. Did so. They said I had to post here. Did so. Finally I will post back on the user forum!

Speaking as a moderator, I think this forum (Internals) is a good place for this question. rustdoc is part of the Rust toolchain.

4 Likes

… I will re-edit my post of users.rust-lang! If you ever read the The Twelve Tasks of Asterix, it feels a bit like the part when he go back and forth between the various administration offices!

4 Likes

Apologies for my advice. I hadn't checked the user forum for a similar post.

I have no idea about rustdoc or anything but I did compile rustc successfully before. stage0 doesn’t really build the rust compiler, as far as I understand. Instead it refers to the compiler that you use to build rustc, the result of building rustc using stage0 is called stage1. Then afterwards, which is a sanity check and also allows rustc to profit from its own new optimizations and ensures that the exact rustc binary doesn’t depend on the choice of stage0-compiler, the newly compiled stage1 compiler compiles rustc again to get stage2.

As far as I understand, you don’t really need stage2 during development. So stage1 should be the correct choice if you plan to modify the source of rustc and want to try the modified rustc out afterwards. Regarding size, you can save on the repository by doing a shallow clone, AFAIR also manually shallow-cloning the submodules (you can find the correct commands somewhere if you google them) saves space (and time!), too, compared to what x.py does on its own. I don’t remember how much space compilation took, but 43GB sounds plenty to me - I would be surprised if that really is not enough.

I would be surprised if that really is not enough.

The 43 GB is after a full clone. After running ./x.py build -i --stage 0 src/tools/cargo src/tools/rustdoc I have only 13 GB left (so it seems that stage 0 takes 30 GB). :frowning: This seems humongous.

What are the components that I need exactly? Is it exactly src/tools/cargo + src/tools/rustdoc?

There is something strange with the -i flag. Starting from 0 (after ./x.py clean) with -i it consume more than 43GB to compile the stage 1, but without it consumed only 13GB (witch is in line with what I was expecting).

-i is the flag which enables incremental compilation which trades disk space for faster subsequent builds. In order to do that, rustc saves additional metadata to disk which can be quite large especially for big crates like some parts of the compiler. If you're extremely constrained on disk space, you shouldn't use -i.

Edit: I see there was more discussion on this here

Thanks. I had the debug option activated in config.toml. Without debug the space taken is totally reasonable. And 43GB isn't what I would consider "extremely constrained".

The question about rustup is still open.

You should almost never need to build cargo when hacking on rustdoc (or rustc). The bootstrap code always uses the version downloaded in stage0 (the most recent beta). If you use rustup toolchain links (see here), rustup has special-case code where commands like cargo +stage1 build will use the cargo from the nightly toolchain if it is not installed in the stage1 toolchain, but rustc/rustdoc will still be "stage1", so it should usually "just work" without cargo being built.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.