Setting our vision for the 2017 cycle

Given that impl Trait for arguments is very possible for 2017, I find the discussion on generics syntax to be on topic, but of course should be focused on adding sugar instead of breaking changes.

1 Like

Some features (like intrinsics) are intentionally perma-unstable. We are planning to completely change the way they are exposed in the future, intentionally breaking all nightly crates that use them. That’s why they are unstable.

Another summary. As before, if I’ve missed or misrepresented anything, please let me know so I can fix it ASAP.

  • Rust should provide a more seamless FFI story root 16 38 44 48 49 50 53 98 99 104 112 122 127

  • Rust should have a lower learning curve (as well as being learnable as a first language) root 4 6 9 54 63 104 112 118 81 101 119

  • Rust needs to reduce developer dependence on nightly (custom_derive, SIMD, etc.) 9 62 92 93 96 97 109 118 129 143

  • Rust needs better answers (either guides, outreach, or new features) for teams & programmers coming from OOP – this seems very related to “Rust should have a lower learning curve” to me, but could be distinct if it covers type system features for inheritance? 3 6 8 14 26 54 90 112

  • Rust needs a better story for embedded and low resource development (re: no_std, alloc failure, etc.) 9 28 45 57 76 129

  • Rust needs cleaner integration with existing tools (system toolchains, build systems, etc.) 30 33 104 105 112 122

  • Rust needs to have a more expressive type system. (type level values, CTFE, variadics, HKT) 48 52 95 99 107 128

  • Rust should be well-equipped for writing servers root 9 66 68 92 104

  • Rust should have basic IDE support root 9 63 104 118 119

  • Rust’s compiler should be significantly faster root 53 77 87 93 104

  • Rust needs to improve discoverability of community projects and initiatives which address our goals. 51 96 104 125 137

  • Rust should have 1.0-level libraries for concurrency and parallelism root 9 92 104 112

  • Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates 17 37 97 129

  • Rust needs clearer, more consistent branding. 62 104 109 106

  • Rust’s core ecosystem should be maturing, and possibly offer a more batteries-included experience. root 96 104 99

  • Rust needs to be more mature and present a more mature image to potential adopters. 104 109 104

  • Rust’s community should provide mentoring at all levels root 81 104

  • Rust needs 1.0-level libraries for scientific computing (matrix manipulations, BLAS wrappers, vector functions) – see Eigen and mshadow 52 135

  • Rust needs better error handling in production. 129 74

  • Rust needs better testing and doc generation across conditional compilation directives 16 24

  • Rust needs to encourage better interoperability between crates which occupy the same domain. 135 138

  • Rust needs to fix existing bugs and suffer fewer regressions. 64 97

  • Rust should improve its syntax for usability of common cases. 55 119

  • Rust should increase the stability and utility of its infrastructure and automation. 68 80

  • Rust’s community and processes need better visibility to “casual” participants and newcomers (e.g. RFC visibility and dashboards, casual is a bad word for this, but I can’t think of a better one here) 9 61

  • Rust’s tooling needs to be first-class to aid adoption (beyond IDE support, i.e. rustdoc) 16 104

  • Rust development should be fundable by the community. 16

  • Rust needs a clearer specification of its semantics. 112

  • Rust needs to continue focusing on what we’re doing well (performance, incremental improvements, core tools, docs, bugs & soundness issues, having a great community). 104

  • Rust needs to continue improving ergonomics. 104

  • Rust needs to define and execute better processes for improving documentation and guides across the ecosystem, but especially for core projects 21

  • Rust needs to encourage more “first contact” usage. 104

  • Rust needs to grow a new temporary team structure (strike team, accelerator, etc.) 70

  • Rust needs to grow its support for platforms other than x86 (and beyond ARM too) 28

  • Rust needs to have data-driven feature prioritization. 72

  • Rust needs to improve its distribution story for environments without internet access. 48

  • Rust needs to simplify existing features, and potentialy remove ones that introduce complexity. 67

  • Rust needs truly first-class Windows support. 112

  • Rust should track adoption metrics. 104

  • Rust’s ownership system needs to aid in modelling more complex systems (i.e. parent/child communication) 3

17 Likes

First: There are over 150 replies to this thread so far. I've skimmed them and nothing jumped out that I felt I was duplicating, but if I have, my apologies. Please point me to the correct existing conversations.

MOVING ON:

Ultimately, the goals listed in the original post here feel like "How does Rust compete with Go", instead of "How does rust replace C and C++". That worries me. Mostly because I'm in that vague "systems programming" space, and I want a better language; but also because I work closely with people impacted by all the easy-to-prevent security vulnerabilities in today's OSes and devices.

At first blush, the list of non-goals seems like the laundry-list of "things that would help get Rust code into places currently using C and C++":

  • Making Cargo work well in more contexts (e.g. big existing build systems)
  • Adding compiler/platform targets
  • Producing more API bindings for core platform libraries
  • Improving our story for programming embedded systems/OSes

I worry, I guess, that the direction of the project is being overly impacted by comparisons laid forth in that orange forum that many of us spend time on. There's an absolute bias there towards web-focused development and SaaS applications, which doesn't necessarily map onto Rusts strengths.

Rust, in my mind, can do the most good in embedded, OS, and other "systems" development. There are already very good languages for writing fast, correct servers. Why push Rust in that space, when it has an opportunity to stand on its own in an area where there's been no serious competition for decades?

16 Likes

You may have misinterpreted that heading. It was listed as "Other possibilities, or potential non-goals" -- the emphasis was supposed to be on the first part ("Other possibilities") and not the second part ("non-goals"). In other words, these are things that seem very important -- are they important enough to bump to the top of the list?

There have been several replies in this area. I'd like to discuss the needs of the "embedded" space a bit more deeply. I am thinking of opening a separate thread though for that so I'll hold off on saying much just this second.

So. There's a lot here. Yesterday and today I sat down and re-read through all the comments on this thread. Let me say up front: @dikaiosune, your summary comments are invaluable, thank you so much for that!

One thing I noticed is that a lot of the ideas here already fall under some of the existing priorities (which is great). In many cases you can see that from the summary that @dikaiosune wrote, where he linked things to [root]. I think there are also a lot of cases (e.g., "Rust needs to continue improving ergonomics") that he didn't link to [root] but which (to my mind) are already covered (e.g., under "improve learning curve").

Digging deeper into some areas

In terms of moving the discussion forward, I wanted to try a thing. I branched off a thread to dig deeper into the needs for embedded development. I would like to see threads branched off for a few other areas that I think merit some deeper attention:

  • Productivity: learning curve and expressiveness
    • I think we all agree this is important, but I've been wanting to start some threads trying to expound on the right ways to do this, and in particular with a focus on "positive sum" outcomes where we don't pit new vs experienced users against one another
  • HPC / numerical computing.
  • Seamless FFI.
  • Game development, perhaps? I didn't see a lot of citations here.
  • Windows development needs (I hear mixed stories here?)
  • Dashboard and visibility
    • visibility into the state of things has been very much on our minds, and part of the point of the roadmap process was to help lay the framing for that
    • @dikaiosune and @brson have been doing great work here with http://rusty-dash.com/
  • First contact: @nrc mentioned ways to make it easier to experiment with Rust and cited React/Angular, I'd like to hear more about what he has in mind

I am going to try and fork off some of these threads, but I may not get to all of them today. Feel free to start up a thread yourself. I think the main thing is that it would be really helpful if we can focus on the most burning problems and also on categorizing the space. A lot of domains (cough HPC, embedded cough) are really enormous and varied. What is a blocking requirement in one case may be a non-issue in others, and it's helpful to clarify some of these distinctions for us noobs.

One thing I want to say: I feel like these threads are useful irrespective of whether we add some text about that particular topic to the roadmap. It's awesome to dig into these areas in depth.

Possible non-goals

Now, one thing I also want to bring up are non-goals. I did see a number of topics raised that I think don't rise to the level of "Rust project goals". There are a couple of different reasons for this:

  • Often, the idea itself may be sound, but it is presented without motivation. I think we really want to focus on the problem we are trying to solve.
  • Sometimes, the idea doesn't seem to require "community focus", but might better be pursued independently.
  • Finally, some things just don't seem as pressing as others.

Here are some candidates that I found. I'll just quote from @dikaisune's summary:

Rust needs to have a more expressive type system. (type level values, CTFE, variadics, HKT)

This is a classic case where I think we need more motivation and less mechanism. I don't want (e.g.) HKT for HKT's sake. I want to know what we plan to do with HKT (which may well affect the precise design of it). Basically I think these kinds of language extensions don't belong on the roadmap, but the problems we aim to solve with them may. Anyway, my take on some popularly floated proposals:

  • const generics and CTFE seem like a valuable avenue for unlocking many high-performance applications (e.g., HPC, small vec, embedded spaces); we should aim for something minimal. @eddyb has already been pursuing a lot of the important refactoring as well of course work by @scott and others on miri.
  • HKT: the application heres are potentially widespread, e.g., Iterable traits, being generic over Rc vs Arc, but HKT as found in Haskell may not be the best solution (and raises a lot of open questions). I like RFC 1598 by @withoutboats quite a lot. It's not HKT, but it's a relatively small, natural extension to the language that can express HKT, and would address things like the Iterable trait and so forth. Not sure how to prioritize this: digging more into the applications may help!
  • variadics: the primary motivation to pursue variadics for me has been making it possible to implement the Fn traits. I think we can do this in other ways (e.g., adding syntax like impl Fn() for T) that would let us punt this a little further down the road. Given that I would want said syntax anyway, I'm sort of inclined to do that. =) (But maybe a good variadic solution is not so much work, not sure). In any case, I'm not sure just where this falls on the roadmap tbh, but I could imagine more power vis a vis Fn traits affecting a lot of areas a little bit (e.g., @carllerche raises this point to me regularly).

Rust needs to have data-driven feature prioritization.

The basic idea here is polling for what features people need etc. I am very much in favor of trying to get numerical data on what people are using and also trying to find out what people want. But I would not want to drive our decisions based on a purely numerical basis. For one thing, good polling is hard (citation: world events) -- as soon as you turn "polls" into (effectively) "votes", you raise incentives for people to do things like twitter-based vote drivers for their pet feature and so forth. I would rather that we try to evaluate things on the merits. But I do think that broad-based surveys are really useful for bringing things to awareness that were flying under the radar.

Rust needs a clearer specification of its semantics.

I think that as part of servicing other goals -- e.g., mature ecosystem and tooling -- we should focus on specifying the areas most urgently needed: unsafe code guidelines and grammar. Trying to formalize and specify all aspects of the type system seems like a big endeavor with very little payoff. It doesn't seem to rise to the level of a roadmap priority for me.

That said, I am very interested in making it easier to be more formal in RFCs. We've done some academic collaborations (e.g., this formalization of the trait system done at UBC and of course the ongoing Rust Belt project), but these efforts tend to lag and subset the language. What is needed to help with RFCs is something much more complete. I think improving and simplifying our trait implementation would be a great starting point. I've been toying with ideas here and would love to dig into them (ah so many half-written blog posts).

Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates

This seems important, but I don't know that it rises to the level of a roadmap point. Basically, if we find a good way to do it, I think we should just do it. But is hashing what's really preventing Rust from making in-roads? (I guess maybe?)

5 Likes

Oh, things I learned recently: If you want to start a new thread based on an existing post, you can click on the "age indicator" (the same way you would get a permalink) and click on "+ New Topic". This gives better metadata to discourse (which of course maybe we can employ for rustydash someday....). Know your tools! :smile:

1 Like

Plugins are definitely a life changer, but other than what's already been said I don't see anything to add.

I agree. Modelling DAGs in Rust is definitely not fun.

And lexical lifetimes…

Moreover, the dispatching can be made easier using my ifunky crate. A higher-level crate could combine it with a cpuid crate to do something closer to GCC function multiversioning (as opposed to bare ifuncs), though without the nice "automagically use the additional CPU features during optimization without requiring separate compilation."

An RFC based on this discussion has now been opened!

4 Likes

One thing we do miss to really pitch Rust in our workplace is a good coverage story.

Right now it’s a wonky experience, and that’s something we’d need to push forward rust components in our stack (distributed servers, one of the main focus if I trust the RFC). I remember stating the same during the Rust survey, but since it wasn’t a default choice, there’s a big chance many people didn’t think about it.

We’re a linux shop so we’re not worried about lacking windows or OSX support, but I kind of get the feeling such a feature wouldn’t see the light of the day without a full support from tier1 platforms. Am I mistaken? I’d be willing to spend time getting that right for the *nix part.

I’d love to see built-in coverage support as well, to go with the built-in test support; a binary built with rustc --test could include built-in coverage analysis. That would make it easy to analyze the coverage of tests written using #[test] methods.

Built-in coverage analysis could do a much more accurate job than external coverage analysis, since it could work by statements/expressions rather than by lines.

3 Likes

I very much agree that Rust programs need to be able to recover from OOM. Returning Result from any function that may allocate would both have terrible ergonomics and massively break backwards compatibility, so the only reasonable thing to do in such circumstances is to panic. That, in turn, means that unwinding must never itself allocate.

It is expected that these panics, like any other, would be caught at rather high levels in the program, and handled generically, like any other unexpected error.

  • return an error code
  • 500 Internal Server Error
  • DB transaction rollback
  • save state to disc, then shutdown or restart
  • unload the image file
  • drop an RPC connection
  • kill the browser tab
  • etc

Or that unwinding must allocate a bounded total amount of memory, which Rust could reserve for that purpose.

1 Like

User defined hooks can be registered which will run when a panic happens.

Not to mention that user-defined Drop impls are called during unwinding.

A band-aid for that would be a lint that triggers if any function is called that may allocate. There has been some work on a #[deny(allocations)] lint by @llogiq

3 Likes

I like this idea. I think it should be Warn-by-default for destructors, but Allow-by-default elsewhere.

Note that if the issue is formatting the message, it is perfectly okay to give a generic message like “Out of memory when formatting panic message” (which can be in static storage) in that case.

Rust should have basic IDE support

I know that when I was first learning computer science having a working debugger in an IDE than I could step through my code and watch variables change over time really helped things "click" for me. As an intellij-rust user, there's not yet debugger integration.

I'm curious if any of the planned work in the Rust community would make it easier for IDEs to have first-class debugging support or if at this point the work is mostly on the individual IDEs?

1 Like