Setting our vision for the 2017 cycle

Setting our vision for the 2017 cycle

Recently we’ve been discussing a roadmap process for the Rust project. Part of that process is setting a project agenda each year by identifying 8-10 key problems we want to try to tackle across the project, which taken together represent a major increment in Rust’s evolution. This post lays out some preliminary thoughts on vision statements for the 2017 cycle.

Overview

There’s no end of possible improvements to Rust—so what do we use to guide our thinking?

The core team has tended to view things not in terms of particular features or aesthetic goals, but instead in terms of making Rust successful while staying true to its core values. Part of that success is for Rust to be increasingly used in production settings, but how much and for what things is something we can and should debate on this thread. More on the core values a bit later.

Rather than trying to identify some particular killer app(s) and drive toward them, our strategy has primarily been to knock down the clearest impediments to Rust adoption that are in front of us, see what doors that opens, then rinse and repeat. The 2016 survey provides a trove of valuable information here—it’s essential background for this discussion—and it directly informs several of the proposed problem statements. These proposals represent the areas we’re most confident need attention this year.

With that out of the way, onto the initial proposals!

In one year’s time:

  • Rust should have a lower learning curve
  • Rust’s compiler should be significantly faster
  • Rust should have basic IDE support
  • Rust’s core ecosystem should be maturing
  • Rust should have 1.0-level libraries for concurrency and parallelism
  • Rust should be well-equipped for writing servers
  • Rust should provide a more seamless FFI story
  • Rust’s community should provide mentoring at all levels

Below are brief elaborations on each point.

These vision statements are deliberately a bit vague. It’s clear that there are important problems in all of these areas, but the precise problems to focus on, their relative priority, and crisp, realistic goals for the year are all open questions. The list should likely grow a bit as well. Our hope is to have a robust, community-wide discussion oriented mainly around the problems, ultimately leading to a more crisp set of vision statements with rationales and goals. After that discussion, we’ll make a revised version in RFC form and go from there (assuming that the roadmap process RFC is accepted, of course!)

Rust should have a lower learning curve

Rust unique features make it valuable, but also means there’s a lot to learn. A common refrain is “the first couple of weeks are tough, but it’s oh so worth it.” How many people are bouncing off of Rust before they get through those first couple of weeks? How many team leads are reluctant to introduce Rust because of the training needed? 1 in 4 survey respondents mentioned the learning curve.

Some potential avenues: improved docs/training materials; improved compiler errors; language improvements (e.g. things like non-lexical lifetimes).

Rust’s compiler should be significantly faster

Bottom line, the edit-compile-test cycle in Rust takes too long, and it’s one of the complaints we hear most often from production users. We’ve laid down a good foundation with MIR (now turned on by default) and incremental compilation (expected to hit alpha this week). But we need to continue pushing hard to actually deliver the improvements.

Depending on how far we want to take IDE support (see below), pushing incremental compilation up through the earliest stages of the compiler may also be important.

Rust should have basic IDE support

For many people—even whole organizations—IDEs are an essential part of the programming workflow. In the survey, 1 in 4 respondents mentioned requiring IDE support before using Rust seriously. Tools like Racer and the IntelliJ Rust plugin have made great progress this year, but compiler integration has yet to get off the ground, and there’s a lot of room to do more.

Rust’s core ecosystem should be maturing

Another major message from the survey and elsewhere is that Rust’s ecosystem, while growing, is still quite immature (1 in 9 survey respondents mentioned this). Maturity is not something we can rush. But we should think about steps we can take to build more momentum around maturity, and think about what “maturity” means to us (e.g., that core functionality exists, is easy to find, and is of high quality).

Some potential avenues: discoverabiliy/curation; 1.0 releases of “core” crates; surveying “batteries included” stdlibs for gaps in Rust’s ecosystem; tooling and guidelines to encourage crate quality.

Rust should have 1.0-level libraries for concurrency and parallelism

Rust’s potential for fearless concurrency across a range of paradigms is one of the most unique and exciting aspects of the language. But we aren’t fully delivering on this potential, due to the immaturity of libraries in the space. The response to work in this space, like the recent futures library announcement, suggests that there is a lot of pent-up demand and excitement, and that this kind of work can open a lot of doors for Rust. In general, then, getting our concurrency/asynchrony/parallelism story in 1.0 shape seems like a particularly important segment of the ecosystem to focus on.

Rust should be well-equipped for writing servers

Shifting gears somewhat, probably the biggest area we’ve seen with interest in production Rust is the server, particularly in cases where high-scale performance, control, and/or reliability are paramount. At the moment, our ecosystem in this space is nascent, and production users are having to build a lot from scratch. Investing in the server ecosystem could make a huge impact this year. But we’ll need to talk through more detail about how, where, and what to invest in.

Rust’s should provide a more seamless FFI story

An essential adoption vector for Rust is to use it for building components in a larger system. For this use-case to really sing, we need a seamless “FFI” experience. The meaning of “FFI” here is potentially quite broad—from making it trivial to bind unsafe C APIs (#include somelib.h for Rust), to C++ integration, to embedding in languages like Ruby, Python or JS, to compiling to wasm. We’ve been making progress on all of these fronts, but to make an even bigger impact, we could consider choosing a small number of such scenarios to really focus on as a community this year.

Rust’s community should provide mentoring at all levels

The Rust community is awesome, in large part because of how welcoming it is. But we could do a lot more to help grow people into roles in the project, including pulling together important work items at all level of expertise to direct people to, providing mentoring, and having a clearer on-ramp to the various official Rust teams. Outreach and mentoring is also one of the best avenues for increasing diversity in the project, which, as the survey demonstrates, has a lot of room for improvement.

Other possibilities, or potential non-goals

We’ve tried to keep the initial list conservative, containing items that the core team is confident are of utmost importance this year for the project. But there are lots of additional possibilities, e.g.:

  • Making Cargo work well in more contexts (e.g. big existing build systems)
  • Adding compiler/platform targets
  • Producing more API bindings for core platform libraries
  • Improving our story for programming embedded systems/OSes
  • More robust project infrastructure (CI, nightlies, Crater, performance metrics, dashboards)
  • More systematic outreach/evangelism
  • A revamped web presence

What else is missing? And, in an effort to focus on the most important priorities, what can we say should not be on the agenda this year?

A word on core values

Part of what makes Rust so exciting is that it attempts to eliminate some seemingly fundamental tradeoffs. One important such tradeoff is between safety and speed. Rust promises

  • uncompromising reliability
  • uncompromising performance

and it delivers. But are we compromising something else? Are we paying with decreased productivity?

I think we’re all proud of Rust’s ergonomics and modern tooling. But can we do better? Are there places where we are forcing explicitness that don’t really buy you much? Can we build libraries that make it easier to rapidly prototype? Can we improve ergonomics while keeping Rust code robust and fast by default? These kinds of productivity gains can also pay dividends for learnability. They can lower the apparent cost of using Rust. Taken together with goals around learnability, tooling improvements, and ecosystem maturation, they lead to a possible overall theme for the year (and ambition for Rust):

  • Rust: safe, fast, productive—pick three.
42 Likes

This is really well considered @aturon, and I think does a great job of summarizing sort of what’s “in the air” w/r/t the direction of the Rust project. I’ll have some more detailed thoughts later about each of your suggested problem statements, and maybe propose some slight changes, but I for now I wanted to make a few off-the-cuff observations.

I really like the way you’ve formulated the “problem statements”, as required by the RFC. These are phrased aspirationally, with a positive spin, instead of the negativity implied by “problem statement”. You even call them “vision statements”, which I think works ok.

I’d like to end up with enough direction that it can apply to the entire team structure (except mods), so each team has some sort of mandate and direction. It’s going to be hardest for the docs and community teams, but I’m glad you’ve already got ideas here that serve them, and I hope they will chime in and help identify what they think is important. I’m particularly eyeing “Rust’s community should provide mentoring at all levels” and suspecting that we can do some productive critical thinking about it.

2 Likes

A few things under the learnability front that I think should get some more attention:

  • It’s incredibly hard to represent single inheritance. Things like specialization and the fields in traits proposal will help with this some, but Rust essentially makes you change the problem statement to fit the language in this regard. Things like Any not allowing you to go to trait objects mean that even if you figure it out, there’s still patterns you can’t represent.

  • It’s not straightforward to build hierarchies of services and components where children need to ask their parents for things. It’s doable, but you have to use RefCell or more complicated patterns. A parent can know about its children. A child can know about its parent. But both at the same time is very hard.

5 Likes

I have been thinking about this recently. There are some great learning resources out there:

However, I feel that that The Book and Rust by Example target an audience that is somewhat new to programming in general (less so for Rust by Example). I have been thinking how we can target experienced programmers. What about a "Rust in a Nutshell"? Something along the lines (very vague, and very roughly):

  • A very quick intro to primitives.
  • A moderately deep (while trying to remain concise) tour to the borrow system. The various differences of copy semantics and move semantics should be emphasized here.
  • A thorough explanation of Lifetime analysis, why it's needed, when it's needed and common tricks to handle common lifetime analysis problems.
  • A through guide in using the Type system.

I know that there is a lot of overlap with what I just wrote and The Book. But, I feel like that if we target an experienced programming group we can make more assumptions to write a more concise guide. Something with a goal like: This is what you need to know about rust. If you know this, then everything else can be found in the STD documentation (which is quite fantastic).

What do you think?

Edit: I call dibs on the title "Rust in a Crabshell"

15 Likes

I definitely think project infrastructure should be one of the priorities, because having good infrastructure helps adoption of Rust. Without a good set of infrastructure tools, I know many won’t even consider Rust. It’s all well and good to make Rust’s learning curve easier, but if Rust isn’t even being considered serious enough to try to learn, it’s not going to help adoption much. There’s a couple small areas I’d like to utilize Rust at work, but it’s just not going to happen without the a good level of stable infrastructure, which makes me sad :frowning: That’s not to say that there’s not already stuff out there heading the right direction, but I do feel it should be part of the vision to get Rust infrastructure more production ready.

Other than that, I definitely like these vision statements

1 Like

I've been thinking that for advanced programmers, we really need a few different guides that take advantage of what you already know. For example, a Ruby programmer is going to come to Rust with quite a different perspective than a Java programmer, who comes to Rust with a different perspective than C++.

I don't think that means we need to have one guide per language, but a guide that makes sense to people with a scripting language background (Ruby, Python, JS, Lua), another for people with background in a 90s-era typed language with GC (Java, C#), and another for people with background in a "zero cost abstract language" (C, C++) would cover a lot of ground.

Also, instead of trying for full coverage of all languages, we could have shorter guides that cover less ubiquitous features of specific languages. For example, we could write a document that helps a Go programmer understand errors in Rust, a Ruby programmer understand traits from the perspective of open classes and refinements, or a Swift programmer understand Option from the perspective of nil and forced unwrapping.

14 Likes

This definitely matches extremely common sentiments we heard in the survey. Can you elaborate on what kinds of tools you mean by "infrastructure"?

This is pretty interesting -- it might be good to talk through a concrete example. I could see this going a couple ways.

  • In some cases, the "right way" to do it in Rust is not to try to mimic inheritance, but rather to use traits in some other way. For those cases, the problem might be best solved by a guide like "If you're used to inheritance, here's what to do in Rust instead".

  • But in some cases, maybe inheritance really is what you need. For that, we really need to finish out our saga of building Rust's ability to express inheritance-like patterns.

Very much agreed! It'd be great to make progress here, either by making this easier to do at the language level, introducing general libraries for it, or writing more docs guiding you on it.

4 Likes

As I mentioned in my One Year of Rust post I think the following priorities should be considered and I wanted to expand them here:

  • RFC’s and the transparency with the community are a mess. Not everyone has the time or inclination to read them over and this leads to things like “When is MIR landing? Is it turned on?” Some RFC’s were accepted long ago but no work has been done on them. Having a dashboard that shows what RFC’s have been accepted, which ones are actually being worked on, and target dates would go a long way for Rust. Having a place to point users would really help people have a clear understanding with what’s coming down the line.
  • Rust has the possibility to replace C and C++ in the embedded systems department but things like #[no_std] don’t work and having to have a ton of workarounds just to get it to work is frustrating. Workarounds doesn’t make embedded dev feel like a first class citizen at all and not focusing on this area of Rust is in my opinion a huge mistake and a large oversight since we’ve been focusing so much on stability and things like MIR. Going forward into 2017 making it easy to cross compile to various embedded systems is one area I think we should really focus on.
  • As for crates we just don’t have a lot of 1.0 crates. Trying to get people to go 1.0 is probably too hard and not something the core team should focus on. However, I think if we rallied the community to stabilize important crates to 1.0 I think would be highly beneficial. Some crates I think we should get the community to stabilize to 1.0 in the next year would be Ring, Diesel, Iron, and Serde for instance. Large projects that provide critical features should be highlighted, helped in creating a 1.0 roadmap, subdividing those tasks into issues, and then letting the community plow through them to stabilize the crate. The hard part will be identifying the crates we should prioritize for stabilization and working with current project maintainers to setup these roadmaps and issues for users to work on.
  • Tooling is really important and while we have an RFC for IDE support accepted no work has been done on it and I think this is something we should work on actually implementing.
  • Custom Derive is something we need badly because too many crates like Serde depend on it and having to use Syntex to get it to work is god awful to set up. Prioritizing this will greatly benefit being able to use larger libraries in production since they would then be able to compile on stable rather than nightly. Too many of the good libraries are on nightly making the use of stable unreasonable and hard if not impossible to use in production depending on company standards.
  • I think we should help new users learn Rust but I don’t really have any good ideas to throw behind this besides saying we need it.

Between all of these and what was already mentioned I think 2017 will have a solid enough roadmap of things to do both with the community and the core team in terms of things we need to do.

9 Likes

I want to post something more substantial in a bit, but until then:

Interesting, I feel like it's very not appropriate for that audience! There is not nearly enough hand-holding for that. We do target people who are new to statically typed languages, or systems languages, but not programming generally.

(Opinions can of course differ; I wanted to communicate how i think of it)

4 Likes

I went back and took a look. I agree, I don't think it's a good book for people new to programming. And if your intentions were indeed geared towards people who are coming from a dynamically typed language I think you nailed it. That being said, do you think there is room for improvement for targeting programmers coming from a statically typed background? C, C++, Java? And if so, do you have ideas for how to approach such a problem? I would be interested in possibly helping with such a project if there are some good ideas.

2 Likes

Can you say a bit more about what you are thinking of when you say "project infrastructure"? When reading what @aturon wrote, I was primarily thinking of things like bors and our own internal CI infrastructure, but it's not clear to me how that would help adoption (except indirectly by making Rust better).

I feel like we had worked out a fairly decent story here at some point, but we've stalled in terms of pushing the various pieces. Specialization and "fields in traits" were intended to be starting points, for example. I've got to revisit the "fields in traits" RFC in any case...

4 Likes

I like these ideas, and also a similar guide mentioned earlier in this thread "If you're used to inheritance, here's what to do in Rust instead" kind of guide. Perhaps my concern with this approach was making such guides accessible. Should we just let these guides appear on the interwebs naturally? Should there be a curated list of guides? What's the story here? And for someone coming from another language, where do they start?

1 Like

If it frees up the core team’s time because more of what they do is automated I could see how that would contribute indirectly to Rust but beyond that I don’t see how it would increase adoption beyond more features being created.

These vision statements are really good, and I think cover all of the important bases.

On learning curve, I think it’s important we also consider significant investment in tooling, i.e. rustdoc. I’ve heard @steveklabnik mention it doesn’t get much love, and that feels like a pity. One example: the Windows docs for std are currently not available online, because rustdoc will only build the docs for the current platform (or really just the currently applicable set of conditional compilation conditions). How are Windows/OS X users supposed to learn std in this situation? Another: how to document module items whose definition depends on conditional compilation stuff? Do I have to copy docs for each platform?

If the current order of vision statement implies prioritization, I would prioritize the FFI story more. In trying to sell Rust, being able to integrate well with already existing/used technology is incredibly important in making an exploration of Rust as a technology low-risk.

One other thing that came up in threads on this forum recently: it would be in an interesting challenge to try to organize a Rust fund to try and help fund the library ecosystem. I came up with a concept around “turning precious metal into Rust”, with a yearly pitch to stated Friends of Rust (I thought up all kinds of puns, too, but forget them) to contribute on the order of 1k/10k/25k to a funding team that would then approve grant proposals from the community. It would maybe hinge on demonstrating the community’s independence from Mozilla, to some extent?

1 Like

Or like making it much simpler & shorter defining an unsecure-but-fast hashing.

This seems a nice and simple idea.

2 Likes

Isn't this short enough? HashMap in std::collections - Rust You just have to use the desired hasher instead.

1 Like

Could you elaborate? What about no_std doesn't work?

It's not easy and not short.