So, possible ways we could end up with a series of small breaking changes over time:
- Bugs or spec version lag in pulldown-cmark implementation are corrected
- CommonMark makes breaking changes between now and 1.0
- "Extensions" currently in use (footnotes, for one, are there others) are standardized in a way that isn't compatible with the current implementation
Let's try to figure out how likely each of these would be.
It's really not that bad that pulldown-cmark isn't right up to the current CommonMark spec. Keeping it constantly up to date is a lot of work when the changes are minor. If it hit 1.0, then there would be more motivation to get it back to passing all the tests.
From what I can tell after the investigation in my previous post, even though pulldown-cmark is not quite up to the current CommonMark 0.27 spec at this time, updating it is unlikely to cause breakage on the same level as going from hoedown to pulldown-cmark could. The fixes just make documents easier to write. Any document updates people make during the conversion are unlikely to break when pulldown-cmark is updated to match CommonMark's latest spec. Passing all the spec tests should help limit bugs, but if the fix is just a bug in the parser, that is probably acceptable. You just tell the person who wrote to the bug rather than the spec that he should have written it to the spec.
So far, so good. What about the gap to CommonMark 1.0?
I wonder if the quote from CommonMark's website is a bit on the optimistic side? Based on this issue, it seems like they previously promised "early 2016" and now we're a year past that.
Can we tell how close they are by looking at commit activity? Commits have slowed way down in the last year or so. That's good, right? However, there are still 30 issues open at the jgm/CommonMark GitHub repository. I'd expect to see more of them being closed or marked somehow if they are closing in on a 1.0. There are a number of interesting threads at talk.commonmark.org as well, but it looks like some of them have stalled out with no decisions made.
Since CommonMark still isn't at a 1.0 level, it could decide to introduce a breaking change before reaching that point.
I think a good step to take here would be to send a stakeholder from this project to get an up-to-date report on the status of CommonMark and the issues keeping it from reaching 1.0. This probably means taking an active role in helping them reach 1.0 as well. If it turns out that some of the undecided items would cause breakage, this would at least help get them decided and into pulldown-cmark before we make the switch from hoedown. If the report is that the only questions left would not break existing documents, we can more confidently move forward.
Ick. This part scares me more.
Good side: If CommonMark gets real extensions or support for arbitrary extensions, it will almost certainly not break existing documents written in CommonMark.
Bad side: There are two extensions already set into pulldown-cmark that were also present in hoedown. Any "official" extension created later will likely interfere with these.
So far, pulldown-cmark includes support for two "extensions" to CommonMark: tables and footnotes. Users can craft these with HTML, but if a markdown version is available, they'll probably use it. They've certainly used them in existing Rust docs. They are turned on by options in pulldown-cmark, and both options were enabled when used to parse Rust docs.
CommonMark doesn't have a built-in way to do extensions. They might get one, but it doesn't seem to be on the road to 1.0. So if we start using a custom extension that later conflicts with one that is added to CommonMark, we could be in trouble. Old tables don't work or we need to support two kinds of tables. Old footnotes turn into something else or aren't recognized. Things like that. Maybe the standardized version of footnotes uses a totally different syntax and we have to keep the old one around forever while also supporting the new one ("Our docs are in CommonMark 1.x with the official footnotes extension, but also you can use an old version of footnotes too, because.").
The GitHub Flavored Markdown Spec includes a Tables extension. I don't know if it is compatible with the one in pulldown-cmark, but they could probably be made to match. Then we can say "CommonMark 1.x with GFM Tables Extension". At least there's a real spec for it that a lot of people will probably use. Switching to it could also probably break existing tables.
What about footnotes? They aren't included in the GFM spec. The footnotes accepted by pulldown-cmark are very similar to those that hoedown uses. There are at least two (one, two) threads on talk.commonmark.org discussing them, but not much is resolved. There is at least one thread at the google/pulldown-cmark GitHub repository that discusses potential changes to the footnote rules used by pulldown-cmark. In that thread, @critiqjo put some effort into a CommonMark-style spec for footnote definitions that could potentially have some legs if it was brought out of that thread to a wider audience. These independent threads seem to share some similarities. Nothing's final yet, but none of those threads agree 100% with the current footnote implementation in pulldown-cmark or the footnotes that are present in Rust documentation already. Most of the differences have to do with nested blocks, a concept that CommonMark wishes to give some uniformity, but hoedown doesn't seem to care about much.