Committing to a Rustfix Tool for All Epochs/Checkpoints Changes

language design
1

A common criticism of some of the recent ergonomics RFCs is that the proposed changes break too much code, or cause too much churn.

In particular, @emk articulated this viewpoint so well on the previous modules proposal (which did involve more churn than the current proposal) that I can't help but sympathize with his concern:

Speaking as commercial user of Rust who's been bringing multiple in-house developers up to speed, and who already has a non-trivial code base to migrate, my feelings about this whole plan are mixed: some positive, some negative, and some wary. This is also the first time I've seen the whole "epochs" RFC, which is apparently a done deal. This is what I get for not reading r/rust all the time.

The whole "epochs" plan makes me very nervous. There are some good points: automatic migration tools, very gradual migrations, and the ability to run mixed-epoch programs indefinitely. But there are also bad points: This means that Stack Overflow is once again going to be filled with obsolete Rust code, and I'm going to have to retrain everybody on the new module system. As far as I'm concerned, this is a case of breaking the "No Rust 2.0 any time soon, Rust 1.0 is stable" promises. Yes, you're taking many steps to ameliorate that, but it's nonetheless bringing flashbacks of the bad old days before Rust 1.0. We absolutely need Rust to be a stable platform, even if that means it's not always perfect. So, my personal verdict on epochs: If the average epoch will involve a change as big as replacing the module system, then epochs are a sufficiently bad idea to make me rethink further investment in Rust. If the average epoch involves nothing worse than a new keyword or something, then the epoch plan is OK, given automatic migration tools and permanent compatibility. I mean, we are used to running cargo fmt on our code regularly.

...

So overall, my personal verdict is: This change is too large, and it would reduce my trust in Rust's stability and fitness for commercial use. If this is a one-time thing with excellent migration tools, then I can grudgingly accept it to improve Rust onboarding. But if the typical epoch is going to make changes this big, I would need to spend a lot more time defending Rust to my colleagues at work. The mere fact that this will break example code on Stack Overflow is a big deal.

Basically, the thought running through the back of my head is "We've had a golden era since Rust 1.0, but is this the end? Will I need to keep up with constant change again?" I simply have too many Rust crates, both at work and personally, to ever chase the upgrade treadmill again. The more I look at the automatic migration tools, parallel epoch support, etc., the more I realize that this isn't an entirely fair reaction. But that's my underlying gut reaction. And that will probably be the gut reaction of some of my colleagues, at least until I explain the whole epochs plan to them, which will take time. Like it or not, those kinds of conversations are also part of the developer experience that's being proposed.

Plus, @glaebhoerl very accurately pointed out that the rust-lang/rfcs comment crowd is probably more likely to support "breaking changes" than production Rust users (even if epochs/checkpoints mean they're not actually breaking anything).

I think it's highly plausible that the demographics of people who participate in discussions on rust-lang/rfcs diverge from that of Rust users in general. I would conjecture that hobbyists and "language enthusiasts" (that is, people like me) are likely to be overrepresented: people who care about Rust to some extent for its own sake, find questions about its design exciting, and basically just want it to be the best language it possibly can be. And that people who use Rust for "real-world purposes" are likely to be underrepresented: people who are interested in Rust merely as a tool to accomplish the things that they actually care about, and whose day probably consists of coming in to work, writing code (which may happen to be in Rust) and doing other work things, and then going home, and where spending time reading and debating Rust RFCs is probably not part of the job description.

And I think the first group is likely to be the most in favor of breaking changes to improve the language, and the second group, having the most to lose, is likely to be the least in favor. But since the purpose of Rust is first and foremost to be a useful tool for accomplishing real-world tasks, we should care more about the opinions of the latter group with respect to this question than the former!

Personally, I spend every weekday maintaining C++ and Javascript code that goes out to tens of thousands of clients in production. I do not use any Rust code at work (mostly because I haven't come up with a sensible place to add any). We're mostly stuck on C++03 and ES5, and have been waiting years for the various infrastructure teams below us to migrate everything we depend on to C++11 and ES6. Some of that migration included us manually changing a bunch of code we owned that was broken by the new standards. So, part of the reason I supported the epochs/checkpoints RFC is that it commits Rust to having a far better "breaking changes story" than either C++ or Javascript have had (even in my limited 2-3 years of personal experience as a professional programmer), which seems really, really good! That's why I don't personally have the concerns described above.

But we could do even better than that, and it sounds like a significant (possibly heretofore underrepresented?) portion of the community wants us to.

Off the top of my head, the obvious ways we could do better are:

  1. Have a stricter policy on what changes epochs/checkpoints can be used for. See @SimonSapin's comment RFC: Evolving Rust through Epochs by aturon ¡ Pull Request #2052 ¡ rust-lang/rfcs ¡ GitHub. I suspect the easiest way to specify this is by doing #2 and saying "only changes that rustfix can do flawlessly are allowed".

  2. Make a firm commitment to automating whatever changes epochs/checkpoints get used for. For instance, we could introduce dyn Trait syntax tomorrow, but we could refuse to deprecate bare trait syntax until a rustfix tool is developed and polished to the point that every production user of Rust can simply run cargo fix to wipe out whatever deprecation warnings they get after rustup update, or run cargo modernize to both fix deprecations and update to the latest checkpoint. Bikesheds abound here, but you get the idea. In my opinion, this would be the best possible story for "breaking changes" that a language could possibly have, short of stagnating completely and refusing to ever change anything.

  3. Commit to not making any more "breaking changes" until the next epoch has actually happened and we've experienced the transition. Just stop at deprecating bare trait syntax and the one path-related deprecation in the latest modules proposal for now. As far as I know, there are no other compelling proposals for "breaking changes" at this time, so this seems like it's likely to happen regardless, but it might help our messaging if we formally agreed to not accept any more epoch/checkpoint-reliant proposals for the time being.

  4. @glaebhoerl's suggestion that "before committing to any large-scale deprecations of existing syntax ... the relevant teams should proactively solicit feedback from the Friends of Rust and any other known potential stakeholders."

I'm making this post now because all of the epochs/checkpoints, modules, dyn Trait and lifetime elision proposals appear to be at the end of the road in terms of optimal design tradeoffs, except possibly for the migration stories, and none have been accepted yet, so we still have time to add stronger commitments on the migration stories if we do want them. So that's my question to everyone watching this forum: Should we commit to an even stronger "breaking changes story" by either specifying what changes are allowed, committing to implementing a seamless rustfix workflow, committing to no more epoch-using RFCs for a while, or something else I haven't thought of?

As you probably guessed from the title of the post, I'm most enthusiastic about #2, but I see no reason we couldn't do all of the above.

12 Likes
2

I think a rustfix or similar tool (your #2) should be necessary but not sufficient. Even with a perfect migration tool, widespread breaking changes should not be made lightly. A tool cannot help migrating code generated by macros, syn/quote, string concatenation, Python templates, etc.

9 Likes
3

This is exactly the plan as I've understood it, but it seems like it wasn't communicated well enough if people don't think so.

I agree with your general point, but its important to note that macro-based codegen (including derives and unstable proc macros) will be hygienic regarding epochs. External codegen tools are more problematic.

4

Another even stronger argument that rustfix is insufficient is documentation and examples, especially ones not part of the official distribution.

1 Like
5

Epoch hygiene is good to avoid breakage when updating the compiler, but presumably any code base under active development may want to migrate eventually. The existence of rustfix should not be enough to skip considering how much impact a given proposed change would have on existing code bases. Some ideas like should be non-starters regardless of tooling. (For an extreme example, consider “replace <…> with […] for generics”.)

6

@SimonSapin @rpjohnst What do you feel would be a sufficient set of restrictions on epoch changes?

7

Can you lay out what you see as the reasons this example is a non starter? (I agree its a nonstarter, but it'd be helpful to get a better sense of your constraints).

8

I wrote in RFC: Evolving Rust through Epochs by aturon ¡ Pull Request #2052 ¡ rust-lang/rfcs ¡ GitHub :

Of course, this RFC isn't suggesting that such a course of action is a good one, just that it is possible to do without breakage. The policy around such changes is left as an open question.

[…]

These downsides are most problematic in cases that involve "breakage" if they were done without opt in. They indicate that, even if we do adopt checkpoints, we should use them judiciously.

I’d like the wording here to be stronger. I worry that the existence of the checkpoints mechanism in rustc is considered a “free pass” for proposing breaking changes. Since this RFC was opened I’ve seen multiple other RFCs and proposals that include breaking changes (to be included in the next checkpoint) like it’s no big deal.

I think this RFC should not leave the policy around this as a completely open question. No need to go into much details, but it should emphasize that making breaking changes are still not a step to take lightly. That they intrinsically cause a burden to maintainers of existing code (even though we take all steps to make that easier to deal with) which should be balanced against the expected benefits of a given change by guesstimating how much existing code would be affected. (For example, code using trait objects is probably more common that code using catch as an identifier.)

This is deliberately a bit vague. I think there should not be a checklist set in stone that says “if you do X and Y then your breaking change is fine”. It should be a case-by-case judgment call, where each proposal/RFC makes some effort at estimating the impact and arguing why similar benefits can’t be obtained without a breaking changes or with a different breaking change that would have less impact.

My point is that a hand-wavy proposal that stops at “we’ll just use an epoch” (or “an epoch + rustfix”) is not good enough.

2 Likes
9

The amount of existing code that would be impacted is too big. Even if rustfix would work in all but a small percentage of situations, a small percentage of a very large amount is still a large amount.

In a less extreme case, this estimated amount should be balanced against the expected benefits of the change.

Edit: even when it can be automated, no one wants to do a migration every two years that affects most files of a project.

3 Likes
10

When you put it like that, it seems so obvious I feel silly for asking the question.

I’d be on board with making “Why does this require an epoch/checkpoint?” a separate section in the RFC template that’s mandatory for any RFC proposing such a change. Maybe tomorrow I’ll try adding that to my dyn Trait RFC.

11

As the author of https://github.com/killercup/rustfix, I believe that having such a tool is very valuable to help with making sure code is idiomatic and reducing deprecation warnings, and even more awesome if it were to support fixes across epochs checkpoints.

At the same time, I’d be surprised if we were able to write auto-fixes for every change in a checkpoint—Rust is a surprisingly complex language with many nuances that make auto-fixes hard to get 100% right.

And just like “Code too verbose? Use an IDE” is not good for language design (IMHO), using a tool to justify breaking changes will make people angry the second it doesn’t work for their niche project generating code using an API definition, or even just when they are suddenly confronted with a 1000 line diff.

3 Likes
12

Swift had syntax migration tool. While it definitely helped to keep up with the changes, it wasn’t painless:

  • Swift’s migration seemed to work in 95% of cases, but that still meant every non-trivial project ended up with some broken code and needed manual fixes. And it was always the worst few cases that confused the compiler, so compiler’s fix suggestions weren’t helping.

  • Migration caused chaos in projects using feature branches. Project-wide syntax changes made pre-migration branches incompatible with post-migration branches. After migration you can’t cherry-pick or revert from old code, and you can’t even run the migration tool again on such mixed-version syntactically invalid code.

  • The tool obviously couldn’t fix code it couldn’t see, so it missed commented-out code, files that were unfinished/temporarily disabled, helper scripts outside the main project, etc.

  • Swift required migration of the whole project in one go, but such instant project-wide switchover was too hard to coordinate in large projects (where different components were maintained by different teams on their own schedule).

Fortunately Rust is easier to split into smaller creates (with potentially mixed epochs), so some of these problems won’t be as bad, but it’s good to keep in mind that automatic syntax fixes are not panacea.

7 Likes
13

I think rustfix should be a tool that fixes errors (like clang’s fixit) or deprecation warnings (other types of warnings are hard to fix in an automated way).

The issue with ‘migration tools’ is that they are necessarily a single step. If every increment of the epoch can only remove features which were deprecated (with alternatives) in the previous epoch, then all code can slowly remove deprecated features over time and once deprecation free, bump the epoch painlessly, since it’s guaranteed to compile cleanly with the same semantic meaning.

1 Like
14

Even so, having a tool to automatically switch from the deprecated features to the non-deprecated ones can make for a much nicer user experience than having to go through the code manually.

IMO, the existence of a tool should be necessary but not sufficient to justify a change. As @kornel described, even a ‘perfect’ migration tool (one that never required manual fixes) would not be a panacea, because of version control and things like commented-out/disabled code (and other reasons). Thus, a tool is not sufficient to justify otherwise highly disruptive changes; it should only slightly decrease the bar for disruptiveness, if at all. However, if a change is going to be made, the fact that a migration tool is feasible in Rust means that it’s vital to provide one. Not doing so feels like sloppy UX, literally from an emotional perspective: it doesn’t feel good to be made to do work you know the computer could do for you. Conversely, it feels great to have the computer do work you thought you might have to do yourself!

2 Likes
15

I’ve asked this in the RFC thread, but it did not get much attention there, so I’ll repeat here: had we considered solving at least some of the issues by improving compiler diagnostics?

For example, to address module paths learnability, we could have rustc try to look up unresolved items in the crate root scope and if module::Item can be resolved from there, emit a note informing user about that, along with suggestions on how to fix it: either use an absolute path (::module::Item) or import module into the current scope (use module).

4 Likes
16

I wrote about this as well (https://github.com/rust-lang/rfcs/pull/2121#issuecomment-324736184).
There are very few diagnostics addressing the confusion points mentioned in module RFCs (and maybe that is the primary reason of why “module system is too confusing”?).

2 Likes
17

Rustfix should obviously be tested against the widest range of code out there - perhaps by integrating with crater:

  1. Build the old code.
  2. Run rustfix.
  3. Build the new code.
  4. Compare.

A release candidate of rustfix could also be available for some period of time - before proceeding with deprecations - to allow parties with closed source code to test it and leave feedback.

18

I believe that was the entire point of my post.

19

Another big point against breaking changes is the teaching story. My university still has some teaching materials from 2004. Some of the screenshots are from netscape navigator! If they ever adopted Rust, they’d probably not update it for years to come, which means that students will never be able to get in touch with “modern” Rust if it changes significantly every 2-5 years.

Now you can argue that there are universities which still teach pascal, and that its more about the concepts, but that’s not the point. Students won’t start projects in pascal, they’ll (hopefully) start projects in Rust. And which Rust will they use? They one they’ve learned, following the path of least resistance. I don’t think this is of any help for consistency in the ecosystem.

Rust is a young language and is still very consistent. Any breaking change will lead to inconsistencies, so I think the cost of breaking changes is very very high, because it destroys this high degree of consistency, even if it just involves running an automated tool.

Also, Rust is not Swift. Swift is a language primarily designed for apple devices. Apple has enough control over the ecosystem to force coders to port their codebase to more modern versions of swift. They can just refuse support for old swift in newer versions of Xcode, and refuse code compiled with older Xcode versions to be submitted to the app store. Outside of app store apps, Swift is practically dead. So doing breaking changes while maintaining consistency is very easy for Swift. For Rust, its very hard. Rust is and will be used inside a much more diverse set of environments than Swift.

About requiring rustfix for any breaking change, I’m wondering how macros 2.0 will fit in there. I don’t think it will be possible to write a migration tool to migrate macro_rules! macros over to macros 2.0 that have exactly the same behaviour.

6 Likes
20

The other side of this is that because Rust is young if something needs to be fixed eventually, now is the best time to do it before the inertia (and cost) is even greater.

2 Likes