Pre-RFC: rustdoc-specific feature flags


#1

(This was supposed to be a less-structured thread about this idea, but then i grabbed the RFC template and started writing anyway, oops! There are still a few holes in the text that can/should be filled in; that’s where this thread comes in!)

Summary

Over its existence, Rustdoc has added several new features that have undergone periods of being unstable as they were being developed or tested “in the wild”. However, nearly all the existing features have been able to use some existing mechanic to tap into to create their instability, using code borrowed from the compiler or by inserting code into to the compiler to perform the check. This document proposes that rustdoc track its unstable features itself, to divorce its stability tracking from the compiler’s.

Motivation

Unstable features in rustdoc are generally “activated” with one of three methods:

  • The use of a special #[doc(...)] attribute command that affects the item it’s attached to (or the whole crate if applicable).
    • These are paired with full #![feature(...)] gates like all compiler and language features, and are checked in libsyntax alongside all the other feature gates, even if this check is the only part of the feature that extends outside rustdoc.
    • Features that use this method include #[doc(include="...")] and #[doc(cfg(...))], and certain std-specific items like #[doc(alias="...")] and #[doc(spotlight)].
  • The use of a special command-line flag which toggles some behavior for rustdoc or provides extra information.
    • These are only allowed with the use of an additional -Z unstable-options flag, which uses the code borrowed from the compiler to verify that it’s being used on a nightly version of rustdoc.
    • Features that use this method include --extern-html-root-url, --resource-suffix, --themes, among others.
  • For two rare cases, the feature is silently activated when you’re using a nightly version of Rustdoc, and turned off when not.
    • These features perform an ad-hoc check of the compiler environment to determine whether to enable the feature. If the version matches, the feature is run; if not, it’s ignored.
    • The two features (that this author knows of) that work like this are the ability to use error codes on comple_fail doctests and intra-doc links.

The primary motivator for giving rustdoc its own stability tracker are attribute-features that have no significant code outside rustdoc, and the “silent activation” category of features. Allowing these to be tracked specifically by rustdoc allows for semantics where documentation-specific features only need to be checked when building docs, and allowing for greater insight into the use of features that are currently silently activated.

Guide-level explanation

The following proposal is for a strawman syntax and semantics. The exact semantics are open to change.

When enabling some features for Rustdoc, you use a different kind of feature attribute: #![doc(feature(...))] This allows your crate to continue building on stable, but only allow generating docs on nightly. For example, imagine you wanted to use intra-doc links in your crate:

pub struct SomeStruct;

/// Performs an action with [`SomeStruct`].
pub fn some_fn(input: SomeStruct) { /* ... */ }

By adding #![doc(feature(intra_doc_links))] to your crate root, rustdoc will start using this feature in your docs.

What’s the difference between #![feature(...)] and #![doc(feature(...))]? The biggest one is that your crate can still be compiled on stable:

$ cargo +stable build
  Compiling example v0.1.0 (...)
   Finished dev [unoptimized + debuginfo] target(s) in 0.35s

…but attempting to generate documentation will cause it to fail:

$ cargo +stable doc
 Documenting example v0.1.0 (...)
error: #![doc(feature)] may not be used on the stable release channel
 --> src/lib.rs:1:1
  |
1 | #![doc(feature(intra_doc_links))]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: Could not document `example`.

If a feature is available to standalone Markdown documents, it can also be activated by a command-line flag. For example, if you have the following Markdown file:

This is a sample meant to show off the use of error codes in doctests, an unstable
feature.

```compile_fail,E0277
fn some_func<T>(foo: T) {
    println!("{:?}", foo); // error: the trait `core::fmt::Debug` is not
                           //        implemented for the type `T`
}
fn main() {
    // We now call the method with the i32 type,
    // which *does* implement the Debug trait.
    some_func(5i32);
}
```

You can run that doctest with the following command:

$ rustdoc +nightly --test my-docs.md -Z doctest-error-codes

Reference-level explanation

Rustdoc keeps a listing of it’s own unstable features separately from the compiler, so that it doesn’t impede downstream users from compiling the crate in question. When building a crate, the compiler knows to ignore these feature declarations, so that crates can continue to build on stable, even though their docs require nightly. When rustdoc encounters one of these feature declarations, it ensures that it’s being run as a nightly build before continuing.

Features declared as CLI flags are collected in the rustdoc::config::Options struct, at the same time as all the other CLI flags.

Features declared as crate attributes are collected at the beginning of crate cleaning and stored in the RenderInfo struct held inside the DocContext. This allows features to be used during as much of the documentation process as possible. (If it turns out we need them earlier, crate attributes are available immediately after parsing a crate, so they can be collected after macro expansion if necessary. In this case they can still be placed into RenderInfo, as features can still be required during render time.)

If a feature requires the cooperation of the compiler (for example #[doc(include="...")] requires a step to be run during macro expansion so that files can be loaded into a crate’s metadata for cross-crate re-export), it will still require a #![feature] gate to signal the compiler, so rustdoc needs to inspect conventional #![feature]s as well as #![doc(feature)]s.

Drawbacks

This would create a “split” in unstable features for rustdoc, with some features still requiring compiler features (such as #[doc(include="...")]) while other features would only need a “doc feature” to unlock.

Failing doc builds on stable but allowing compilation would create a situation where people can’t create local docs for their dependency tree, because some deep dependency (for example futures, which was an early-adopter of intra-doc links) started using one of these features. In the presence of docs.rs this may be less of a problem, but unless you have a nightly rustdoc handy, this means you can’t have offline docs, a dealbreaker for some.

Rationale and alternatives

The hope with adding a separate means of tracking features is to reduce coupling between Rustdoc and the compiler. As some features still require its cooperation, this may not be totally worthwhile, but at the least, we can provide the means for Rustdoc to handle some more of its own feature selection.

  • Status quo. Keep using compiler feature flags, and force crates that want to use these features to require nightly. As it’s possible to inspect the status of nightly features without requiring that they gate something, the implementation can still allow for gating some of the “silent activations” behind an explicit feature.
  • Same semantics, but still use #![feature]. With some work, libsyntax could be taught to allow certain feature flags if it’s not being run as rustdoc. This requires even more cooperation from the compiler, but unifies the interface between “rustdoc-only features” and “rustdoc/rustc combo features”.
  • ???

Prior art

Rustdoc currently uses the same stability infrastructure the compiler uses for its own language and compiler features. Inspiration can be taken from its design to ensure that any lessons learned from its implementation can be applied to this feature.

???

Unresolved questions

How should we handle features that can be applied to both crate docs and standalone Markdown documents? If the only way to enable a feature is via a crate attribute, then Markdown documents have no means of activating them.

The rustdoc-specific feature that has significant code outside of rustdoc itself is #[doc(include="...")]. Is it feasible to move its feature gate (#![feature(external_doc)]) to one of these doc features, and only execute its code in libsyntax if the feature is set? This would require that libsyntax also start tracking features like this, but would allow crates with doc(include) to start building on stable again, if this was the only feature they were using.

???

Future possibilities

???


#2

Failing doc builds on stable but allowing compilation would create a situation where people can’t create local docs for their dependency tree

Note that it’s not just local docs, many people publish and maintain their own docs (gstreamer, hyper, servo, all do this).

This can be solved by requiring some cargo key that links to a fallback url for docs, so if the docs for a crate can’t be built the other crates can still go on.


#3

Why not use macro namespace ?

#![rustdoc::feature(...)] seem much cleaner to me, especially since current the #[doc] attribute serve a completely different purpose.


#4

The documentation key already exists, but right now that’s just used to seed the link on the crates.io page. Being used as an html_root_url like you’re suggesting requires that it’s the actual doc root. For example, on my own crates i’ve linked to https://docs.rs/crate-name which will redirect to the latest version, so that i don’t have to update the link when i publish a new version. However, this won’t work as a root-url, since (for example) https://docs.rs/futures/future/trait.Future.html doesn’t redirect to the latest version - the proper root-url in this case is https://docs.rs/futures/0.1.25/futures, so that rustdoc can assemble the correct links for it.

That’s probably something to consider, but the #[doc] attribute is already used to configure absolutely everything about rustdoc. You use #![doc(test(attr(...)))] to configure your doctests, you use #[doc(cfg(...))] to give rustdoc conditional compilation information, you use #[doc(include="...")] to splice in a separate file, you use #[doc(hidden)] to ask rustdoc to remove an item from documentation, and so on. Adding “yet another” parameter to the #[doc] attribute is business as usual at this point. Attempting to change that would introduce another split in rustdoc’s attributes, as many are already usable in stable code, and removing those would break those crates’ documentation.


#5

I don’t see any links to the accepted “Attributes for tools” RFC, so: https://github.com/rust-lang/rfcs/pull/2103


#6

Ok I did not know the #[doc] attribute was so heavily overloaded, so there is no need to make an exception for this option.

But I think there should be a separate RFC to move all these options to the rustdoc namespace.


#7

We could probably move the attributes over without an RFC - we would just need to leave support in for the old ones as well, at least until an edition transition (after which we would still have to support them on old editions).