Pre-Pre-RFC: Moving Rust By Example to mdBook, into rust-lang.org

[Havvy]

Some short history. Rust by Example was originally created as a project outside of the standard Rust language project with its own website and development. Later on, the domain and project were donated to the Rust Language project. It has been maintained by the Rust Language project since then in its own repository and keeping its website. We link to it on the main documentation page.

This setup has served us well enough. But it’s not without problems. It’s another domain we have to keep track of. Unlike the main site, it’s not localized or in a state to be localizable. As it looks to be its own domain, it’s not clear that people can contribute to it not does it feel like it has the official Rust documentation seal. It’s also using its own build system (gitbook) which adds to the difficulty of contributing to.

I’m proposing that we try to bring Rust by Example into the rust-lang.org fold. Convert it over to mdBook, improving mdBook where necessary. I’ve brought this up in the last docs team meeting, and others were receptive to the idea. I’m not entirely sure what all work needs to be done, so that’s what I’m asking here. What needs to be done to get Rust by Example into rust-lang.org?

[projektir]

I have found Rust-by-Example to be very useful so far and wouldn’t mind seeing it get more love. It’s the quick reference Rust needs. Putting it more prominently on the official website would certainly help with that.

Converting to mdBook will make it more approachable for contributions already.

The steps I’m currently aware of are:

1. Modify mdBook to support features required by RBE.

• This seems to mostly be the code editor, which I’m working on right now.

2. Convert RBE to mdBook. ?. Clean up some other tech debt?
3. Move RBE to rust-lang.org.

That’s about all I currently know.

[mdinger]

As soon as mdbook gains support for code editing, RBE can/will probably shift. News that someone is working on that feature is good news indeed. RBE has wanted to shift for quite some time.

[steveklabnik]

RBE has wanted to shift for quite some time.

Absolutely! Big

[projektir]

@mdinger

I have some questions as to what are the requirements for RBE to transfer over in your mind.

Currently, RBE provides its own custom wrapper over the Ace editor with extra JavaScript to run the code as well as a button to Reset the code to what it used to be. These look like they can be added to any existing project as long as code blocks are transformed to Ace divs and the user provides the folder with JS and CSS.

Is the main thing to do here is providing some way to generate Ace divs or is something more integrated expected?

This is the issue for adding Ace to mdBook: #247

[mdinger]

There are definitely a couple things RBE really wants from a generator like mdbook:

• verification that the website works correctly at build time
  • testcase examples must build without errors
  • links should not be dead
• editable examples which are runnable
  • their theme should be consistent with the page theme but does not need to be customizable
• useful errors when the website does not build correctly (currently a problem and hard to diagnose)
  • when the *.json file listing the files is malformed, the problem should be clear
  • an included file was given the wrong name or is missing
• internationalization support
• pdf publishing along with other formats

Those are the main things I can think of off the top of my head. I would consider the first two blockers though I wouldn’t recommend blocking because link checking wasn’t implemented. Everything else is probably on the feature list to eventually be implemented.

I think mdbook already tests examples it includes which meets the first point. I think this is important and is part of what complicates the current build setup. Currently, in addition to parsing the webpage into editable and regular webpages, tools are setup to test all the rust files included. Being able to hand this off to the website generator would be ideal.

Editable examples are basically one of the best things about RBE and it definitely cannot lose that feature. If it is as simple as generating Ace divs, I don’t know why you would do it otherwise. I’m not sure what an alternate more integrated approach would entail or what the benefits would be. I would note that not all projects may want to use editable code examples; they might want the option to choose which they want to use. For example, the book has traditionally not had editable examples and they might like the option for it to stay that way. They also might like the fact that non-editable is more lightweight.

Let me know if you need any more input. I’m pretty familiar with the project.

[projektir]

Wow, that’s a lot of stuff. It’s good to record all that here, but in the interest of doing one thing at a time, I’m going to try to focus on the requirements for the editor:

[quote=“mdinger, post:6, topic:5319”]I would note that not all projects may want to use editable code examples; they might want the option to choose which they want to use. For example, the book has traditionally not had editable examples and they might like the option for it to stay that way. They also might like the fact that non-editable is more lightweight. [/quote]

You’re absolutely right! mdBook wants Ace as an optional dependency. And this changes some things, because the way playpens are right now, I will not be able to integrate Ace into them (due to hljs collision). It will have to be added separately, perhaps as another type of playpen entirely.

Editable examples are basically one of the best things about RBE and it definitely cannot lose that feature. If it is as simple as generating Ace divs, I don’t know why you would do it otherwise. I’m not sure what an alternate more integrated approach would entail or what the benefits would be.

Originally, I wanted to fully integrate it to basically keep it in line with how existing playpens work and avoid duplicating functionality.

I was also hoping to let the user basically just set the config value and get editable code, without having to worry about what it looks like or how it runs code. I can still sort of do this, but I’d have to basically grab RBE’s CSS and JS and make it accessible to mdBook (and/or merge it with existing stuff). Otherwise, the user will have to provide the JS and CSS themselves, and that just doesn’t seem useful.

The other but perhaps more minor part is styling. RBE playpens look very different from current mdBook playpens. Large buttons vs small icons. hljs is actually part of that but of course if I put it on top of Ace, Ace will stop working. Maybe this will just be some style debt we live with for a bit.

I think mdbook already tests examples it includes which meets the first point. I think this is important and is part of what complicates the current build setup. Currently, in addition to parsing the webpage into editable and regular webpages, tools are setup to test all the rust files included. Being able to hand this off to the website generator would be ideal.

I’ll have to look deeper into how testing happens. The only thing I’ll ask here: RBE currently uses syntax like {hello.play}, and then some script (I presume, I don’t think I ever found it) looks for those and generates a few divs with Ace and playpen runner.

So I may do something very similar here, but I was wondering how important the {file.play} syntax was to you? Can it just be the raw code inside of ```rust, editable``` or something? Do you have a preference? I think mdBook actually supports referencing files as well (syntax may differ), but inline code seems more standard and I wanted to check how important this is.

[mdinger]

I was also hoping to let the user basically just set the config value and get editable code, without having to worry about what it looks like or how it runs code. I can still sort of do this, but I’d have to basically grab RBE’s CSS and JS and make it accessible to mdBook (and/or merge it with existing stuff). Otherwise, the user will have to provide the JS and CSS themselves, and that just doesn’t seem useful.

I would consider switching the css to whatever ace uses if it needs to change though I thought it was already basically using compatible themes. For example, I know the coal theme on mdbook basically uses the tomorrow night theme which ace supports. BTW, I’d have thought ace supported using highlighting files if you told it which one to use. Is that not the case?

I actually thought mdbook was already setup to send the files to the playpen so the compilation part is already setup and so I’m not sure what you are suggesting regarding the user providing their own JS and such.

So I may do something very similar here, but I was wondering how important the {file.play} syntax was to you? Can it just be the raw code inside of rust, editable or something? Do you have a preference? I think mdBook actually supports referencing files as well (syntax may differ), but inline code seems more standard and I wanted to check how important this is.

The mdbook syntax of {{#playpen file.rs}} is fine. The syntax used in RBE is mainly so it doesn’t grab the wrong types of files. I’d probably recommend something like {{#playpen --editable file.rs}} or something depending on how that type of handlebar notation works.

I don’t think it is really important that the examples be in external files. The reason it is that way is probably because extracting it from an *.md file is much more complicated than from a file directly and the tooling was not already setup to do it.

Historically as I remember, RBE has had the limitation of only supporting a single ace embed on a page meaning extra examples on a page could never be editable. This also meant that their style was different because they were using and internal css setup instead of something more robust. It would have been better for all of them to be using something standardized.

I’ll have to look deeper into how testing happens.

I think it happens right here.

[Azerupi]

Let me chime in here This is long overdue, I know @mdinger has wanted to port RBE to mdBook since a long time. Development was stagnating a little at that moment, but since a couple of months mdBook has been getting more attention and a bunch of awesome contributors. So let’s get this over the finish line!

I will make a tracking issue on the repository where we can track and discuss all the features that need to be added for RBE. I will also create a milestone for it so that we can track progress.

The mdbook syntax of {{#playpen file.rs}} is fine. The syntax used in RBE is mainly so it doesn’t grab the wrong types of files. I’d probably recommend something like {{#playpen --editable file.rs}} or something depending on how that type of handlebar notation works.

There is no handlebars involved actually. I took that specific syntax thinking that I would make handlebar helpers available to the users directly in the markdown. But now I don’t think it’s such a good idea. So the actual syntax is open for change. I do think however that we need to choose a unified syntax suited for all future mdBook plugins / extensions.

The testing capabilities of mdBook are really rudimentary, barely enough for what the TRPL2 needs. They will probably need improvements to satisfy the needs of RBE.

EDIT: Tracking Issue and RBE Milestone

[projektir]

I would consider switching the css to whatever ace uses if it needs to change though I thought it was already basically using compatible themes. For example, I know the coal theme on mdbook basically uses the tomorrow night theme which ace supports. BTW, I’d have thought ace supported using highlighting files if you told it which one to use. Is that not the case?

I was referring to surrounding styling, not really the colors/code. RBE has the active code area, the Ace part itself, the buttons. None of these things are in mdBook right now. Nor is the .css (without which everything will be horribly misaligned). And they’re a very different style from mdBook (large blue buttons vs tiny play icon that mdBook uses).

Examples from mdBook and RBE to see them side by side:

mdBook: https://rust-lang.github.io/book/second-edition/ch12-05-working-with-environment-variables.html RBE: http://rustbyexample.com/generics/assoc_items/the_problem.html

I’m referring to the stuff in the rust-by-example/node_modules/gitbook-plugin-rust-playpen/book/ folder. Something of that nature would need to be added for Ace to be usable in mdBook, so I figured since it already exists it can be adapted.

[quote=“mdinger, post:8, topic:5319”] I actually thought mdbook was already setup to send the files to the playpen so the compilation part is already setup and so I’m not sure what you are suggesting regarding the user providing their own JS and such. [/quote]Yes, but I cannot trivially use it if I add Ace, I’d have to at least tweak JS to grab the right piece of text, at which point it’s likely easier to just use Ace’s provided functions directly, which is different from how non-editable playpens work. Furthermore, mdBook has no functionality or setup for a code Reset, which is part of the JS in RBE.

I think it happens right here.

I meant in mdBook. I haven’t seen how it does testing yet and whether it’s hooked into code samples directly or how. Just haven’t had a chance to look at that stuff yet.

@Azerupi Nice to see you join in!

There is no handlebars involved actually.

Can you clarify this? The renderer file for mdBook is called hbs_renderer and talks about Handlebars here and there. I don’t know if it matters that much I’m mostly just curious.

[mdinger]

I wouldn’t see any problem with using a fully integrated ace and I’m not sure why you’d need RBE CSS. I’d imagine you’d need customization for the buttons but that’s fine.

I don’t think the style of the run and expand buttons is currently very important. The mdbook style is quite clean and acceptable and there is little reason it needs to change. RBE isn’t really tied to that style. It is just one that it happened to get which looked okay.

I might have slow responses for the next few days.

[Azerupi]

Can you clarify this? The renderer file for mdBook is called hbs_renderer and talks about Handlebars here and there. I don’t know if it matters that much I’m mostly just curious.

I mean for the playpen inclusion, the used syntax is {{# playpen file.rs}} which looks like a handlebars tag but is actually not. I just search and replace, so if we want to change that syntax it’s not a big deal. The only thing I would like to consider if we do, is that the same syntax can be expanded later for any other mdBook extension and doesn’t clash with any of the markdown directives.

[projektir]

PR for the Ace editor part: https://github.com/azerupi/mdBook/pull/338

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.