Rustdoc: reStructuredText vs Markdown

[tony]

I use both languages and have considered Markdown. reStructuredText has suited me better for technical writing (especially programming).

Just wanted to chime in and say CMake implements a subset of reStructuredText and Sphinx:

Kitware/CMake/blob/bb9d71b4fee2cc4abf55e0dcbadf85c6cbe0d07d/Source/cmRST.cxx#L21

Kitware/CMake/blob/bb9d71b4fee2cc4abf55e0dcbadf85c6cbe0d07d/Source/cmRST.h#L19

This is in addition to using it Sphinx for documentation.

There’s nothing wrong with implementing reST-based idioms piece by piece. The real power of reST being pivoted toward technical writing as well as having standards, including a specification for implementing custom directives and how arguments are passed into them.

The thing to like is, after you understand how directives work, there’s no surprises. Sphinx is a popular project that pulls in it’s own directives on top of what reStructuredText has out of the box.

[flying_sheep]

how is the current status? still a markdown-like thing that isn’t markdown?

i’d still like to promote rST. its configurable roles are perfect for documentation.

for example look here. you see the different fonts?

filenames, classes, methods, exceptions, … are all visually distinct and (whenever useful) link to the python stdlib docs or internally. this is all done via roles. refs allow you to just specify the section label and compiling the document, it will insert the section name (and you won’t have to adapt such links to changes)

it’s similarly flexible as LaTeX, but the documents are static and only the roles and directives are defined using a turing complete language, not the whole thing.

[DanielKeep]

I’d like to see rST as an option for longer, more complicated documentation. Markdown is serviceable for simple prose, but the instant you need more than basic formatting, you’re on your own.

Maybe it would encourage adoption if there was a pure Rust implementation of rST that could be dropped into rustdoc… the one thing Markdown has over rST is that there are lots of implementations.

[dch]

While I’m not a rust contributor I did implement Apache CouchDB’s documentation and I learned a few things that might be helpful to you.

Most important is to be clear what you’re trying to achieve. - you’ve got a book, a website, cargo - all of these have Requirements. You will need to compromise, so rank and score these. And make a timely decision.

1. What functionality is needed?

• markdown has limited advanced document functionality
• are cross-document references important?
• do you need to extend the syntax?
• do you need the same format across book, website, crates, stdlib?

2. Who is expected to write 90% of the docs? Does this tool/process make this really easy for them?
3. How easy is it for occasional contributors to send in something? Over time this is equally important.
4. Do you need a native toolchain today for this? I’m sure you’ll end up writing your own eventually in Rust of course!

With CouchDB, the community settled on .rst for 4 reasons IIRC:

• we have some top-level python people to fix or implement anything we needed
• the functionality was well aligned
  • the inter-doc links and HTTP/REST-style markup and plugins are a huge win for us
  • what version something was added or changed is easy
  • converting from docbook into .rst was do-able as a 1 off task. Enter pandoc, ditto for .md -> .rst also
• the generation & publishing toolchain is very easy to install on all platforms incl Windows, Linux, BSD
• the people who were going to write the docs could live with the syntax

Note that Sphinx is what we needed, which is not the same as vanilla .rst. This isn’t a big issue, but don’t confuse things.

Finally, people like me who really detest the .rst syntax just wrote the stuff in markdown and used pandoc to convert over before committing #protip.

[flying_sheep]

yeah, i was referring to sphinx as well. nevertheless, rST is what powers it. with plain rST you can use those semantic roles like :math:`\log x^2` or :py:func:`io.BytesIO`, then you need to let your implementation (e.g. sphinx) or extensions you wrote for it (a new role is often like 20 lines in python)

with markdown you don’t have this extensible concept, so you end up inventing syntax extensions that are incompatible with and not seen by other markdown implementations. rST implementations however can e.g. ignore unknown roles or wrap them in <span class="somerole"></span>.

[hauleth]

Maybe we should use AsciiDoc? AsciiDoctor has Markdown compatibility and is designed for writing documentation.

[Gankro]

For those showing up these days:

This thread is super old. Like pre-1.0 when breaking all the documentation was sort of maybe a possibility. 1.0 has landed, we have thousands of pages of documentation using markdown.

It’s done. Markdown won.

Any shiny features we want will almost certainly be grafted on top of markdown, hopefully using the very nice API provided by https://github.com/google/pulldown-cmark

If you’re really interested in improving rustdoc, you should ping @alexcrichton about what needs to be done to land pulldown-cmark in-tree.

[skade]

I’m split here. Nothing would stand in the way of making the format in use pluggable (and many other frameworks in other languages do). While I would never drop Markdown, offering other options is sound.

FWIW, I actually think rustdoc should be rewritten from the ground up anyways…

[vks]

We always have the option to roll out our own Markdown dialect. (I believe that would be similar to what they did when Sphinx was developed for Python.)

[flying_sheep]

nah, converting it all isn’t a problem thanks to pandoc. markdown’s functionality is a straight subset of pandoc, and pandoc supports all of markdown so i doubt very strongly that there will be issues.

the world has enough incompatible markdown dialects, it’s time to actually do something right for once. (note that i only mean “not choosing markdown”: other than that there are tremendously many good decisions in the rust exosystem)

i propose the following:

1. make rustdoc do rST and/or asciidoc as well. both are extensible by default (directives&roles/macros) and therefore adequate
2. add a Cargo.toml field like doc-format
3. emit non-markdown (rST or asciidoc) by default. everyone choosing to keep using markdown just has to add the doc-format key.

i can’t let this rest because of how annoyed i am each time i see stuff like this where links to the relevant classes and functions only don’t exist because markdown is used. that’s simply throwing away convenience that we could have had for free (well, once there exists a proper rST or asciidoc lib in rust)

[hauleth]

There is thing in ASCIIDoc named Markdown compatibility http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/#markdown-compatibility. Only thing that would need to be added to it is Markdown link syntax and it would provide compatibility with 99% of the existing documentation.

[djc]

I’m interested in this because I also have a bunch of experience with reStructuredText, and having found Sphinx quite awesome to work with. I’ve now written some Markdown for rustdoc stuff, and I’ve definitely felt that rustdoc/Markdown is very limited in comparison.

I just noted this thread, which seems to has gone on forever, so allow me to get a bit meta: what needs to happen for this issue to make progress? (Or alternatively, maybe something has already happened since this thread was started?) Should there be an RFC about, let’s say, optionally adding reStructuredText and/or ASCIIDoc formats? Or is the RFC process not followed for this type of thing?

The main objections to reStructuredText seem to be “I hate writing reStructuredText”, sometimes substantiated with examples of how it is too strict or has draconian error handling. On the other hand, it seems clear that Markdown (in any of the common varieties) is not extensible enough. So, it does seem like some change is needed.

[lu_zero]

reStructuredText is overly verbose and complex even for simple tasks (e.g. linking urls), markdown with few extension can provide a better experience without extra limitations.

ReadTheDocs already provides some commonmark-docutils bridge so a markdown flavour can be used in sphinx.

[getreu]

On the other hand, it seems clear that Markdown (in any of the common varieties) is not extensible enough. So, it does seem like some change is needed.

I believe Asciidoctor is the best choice to enhance Rust’s markup:

1. Asciidoctor is Markdown compatible and supports its basic syntax.
2. You can write your scientific articles and thesis with Asciidoctor: (example Enhance Embedded System Security With Rust
3. The Asciidoctor tool-chain can render “.pdf”, “html5”, ebook, slides, … with or without docbook.

[lifthrasiir]

Asciidoctor is Markdown compatible and supports its basic syntax.

If my understanding of the compatible syntax is correct, it crucially differs from the link syntax which is an enough problem to be adapted.

[getreu]

With Asciidoctor migration from Markup is made fairly easy as it supports Markup’s basic syntax. Nevertheless for more advanced features you will find more differences. Most important, people can just continue typing the way they are used to. But it is much more: Asciidoc is another way of expressing docbook semantics making it is a powerful as LaTeX.

[getreu]

If my understanding of the compatible syntax3 is correct, it crucially differs from the link syntax which is an enough problem to be adapted.

@lifthrasiir : I forward you Dan Allen’s comment on this:

“Keep in mind it’s possible to add support for the Markdown link syntax using an inline macro extension.”

[dobenour]

AsciiDoctor is GPL which could be a problem. Sphinx is under a BSD license, which is much less likely to be a problem.

[skade]

AsciiDoctors License is MIT: https://github.com/asciidoctor/asciidoctor/blob/master/LICENSE.adoc

[getreu]

Please have a look at a followup in Rustdoc: Asciidoctor vs. Markdown.