Setting our vision for the 2017 cycle

143

I would like to see a focus on making sure that nightly-only features are either

  1. Deemed not useful after experimentation and removed.
  2. Being actively iterated on towards being “done”.
  3. Promoted to the stable channel.

Right now, it seems that there are features that are clearly useful, are not being actively iterated on but, yet, are stuck in a perma-nightly limbo without getting promoted to the stable channel. Examples that come to mind include #bench, access to LLVM intrinsics and inline assembly. The underpinnings of serde also seem clearly useful but stuck on nightly, but it’s less clear to me if they are being actively iterated on. Useful features being in a perma-nightly limbo seems unhealthy.

Of the above, I care the most about access to LLVM intrinsics as a means of generating CPU instructions that are not expressible in safe Rust. Allowing the Rust ecosystem outside the standard library to use unsafe LLVM intrinsics would make it possible to develop safe abstractions in a way that would allow for more experimentation than developing features for the standard library does and in a way that wouldn’t make the standard library into a bottleneck.

Additionally, I think it would be good to get debugging in the case of the MSVC flavor of Rust on Windows on par with Mac and Linux in order to avoid situations where developers of cross-platform software (like Gecko) are shy to use Rust for areas that might have to be debugged on Windows specifically.

Also, as a matter of being competitive with C/C++ in client-side end-user apps that aren’t compiled for particular in-house/cloud server hardware, it would be good to be able to enable instruction set extensions on a per-function basis. (I’m aware that this is blocked on the underlying LLVM feature. Also, this feature wouldn’t be fully useful without access to LLVM intrinsics.)

7 Likes
144

As frustrating as it is to need some feature and find out it’s unstable, I don’t think there is a good solution to the problem. Prioritizing little-used existing features when there are more important new features waiting to get implemented (here, “feature” includes more than the compiler’s #[feature(..)]) [*] is a bad trade off, and simply removing everything that won’t get stabilized pronto just screws over people who need it and can use nightly, without helping the rest of the community.

And then there’s the fact that some things just won’t get stabilized ever. These are only exposed for the benefit of core, std and other crates coupled to the compiler. It’s not my decision but I suspect access to LLVM intrinsics is among those, since it would effectively require an LLVM backend (bad for alternative compilers) and furthermore the exact set of intrinsics available depends on the LLVM version (which we’d like to be able to update whenever profitable).

[*] edit: I accidentially some words

145

simply removing everything that won't get stabilized pronto just screws over people who need it and can use nightly, without helping the rest of the community

I am not advocating the removal of useful stuff. Rather, I'm advocating the promotion of useful stuff to stable.

If a nightly-only feature is so useful that it cannot be removed, it should be either actively iterated on or be considered ready and be promoted to stable. If promoting it to stable seems clearly inappropriate, please iterate on it to make it appropriate. Leaving it as-is on nightly means the feature will be entrenched on nightly at which point iterating on it will be too painful, so it will need to be promoted as-is anyway eventually (or otherwise nightly becomes the de facto real Rust and stable becomes a hindrance).

It's not my decision but I suspect access to LLVM intrinsics is among those, since it would effectively require an LLVM backend (bad for alternative compilers) and furthermore the exact set of intrinsics available depends on the LLVM version (which we'd like to be able to update whenever profitable).

Maybe "LLVM intrinsics" isn't quite the right term. ISA-specific intrinsics as seen from C/C++ are portable across GCC, clang, MSVC and ISA vendors' proprietary compilers, so exposing them on the same level of abstraction in unsafe Rust as they are exposed to C should be good enough in terms of not painting Rust into a corner in terms of being able to change compiler infrastructure later or to grow independent interoperable implementations. (Though in some cases, it would be theoretically possible to make the Rust exposure a bit nicer, e.g. with NEON intrinsics where the instruction doesn't produce a result in one register but modifies the two operand registers, it would be more elegant if the corresponding intrinsic in Rust returned a pair instead and took two values by value instead of taking to pointers.)

1 Like
146

The problem is resource allocation. There are only so many people actively contributing to rustc, and they have to decide what is the best use of their time. That's what this thread is all about: which 'features' (or things not technically a feature) should those people work on for the next year.

That doesn't stop any sufficiently motivated user from taking a look at the tracking issue for a feature, and contributing patches to take care of the bundle of bugs that feature has accumulated.

3 Likes
147

No one wants these features to be on nightly (except the ones that are guaranteed nightly forever because they are implementation details). There just isn't enough labor available to resolve all of the bugs in all of the features any more quickly.

1 Like
148

That sounds nice but I think it's unrealistic and it could be harmful to have such a general policy. I think instead it's more useful to focus on making progress towards resolving the fate of high-priority features like some of the ones you mentioned, without trying to make a general policy.

Examples that come to mind include #bench, access to LLVM intrinsics and inline assembly.

IIRC, you can do #[bench]-style benchmarking on stable using GitHub - servo/rustc-test: A fork of Rust’s `test` crate that doesn’t require unstable language features.. I agree that it would be useful to see progress on removing the built-in benchmarking thing in favor of everybody using rustc-test or similar things. Since this would be a removal, it seems like a relatively easy thing for somebody outside the Rust core team to lead the charge on.

Additionally, I think it would be good to get debugging in the case of the MSVC flavor of Rust on Windows on par with Mac and Linux in order to avoid situations where developers of cross-platform software (like Gecko) are shy to use Rust for areas that might have to be debugged on Windows specifically.[/quote]

I agree!

This isn't blocked on anything internal to LLVM. I've been doing this in ring since the beginning of time, manually. It just requires writing a bit of C and/or assembly code and doing the per-arch dispatching manually.

149

As long as debuginfo is enabled, debugging is actually really quite solid in the current version of pc-windows-msvc Rust. Local variables, unmangled symbols, types, everything works fairly well. The two remaining issues are getting std distributed with debuginfo enabled (or even better building std locally), as well as taking advantage of Natvis so that you can see the contents of things like Vec and String.

1 Like
150

Since this is a thread that asks for input about resource allocation, I'm suggesting that, as a matter of resource allocation, it's worthwhile to prioritize the avoidance of the unhealhy situation where a feature is perma-limboed in nightly simultaneously in such a good shape that it gets non-trivial use (which is contagious and pulls other crates towards nightly-only taking them away from the pool of crates that a ships-in-Linux-distros-and-therefore-on-stable-channel project like Gecko can use) and in such a bad shape that it doesn't get promoted to stable.

I realize that features getting used but not iterated upon getting auto-promoted could lead to mediocrity. But the thing is, if more and more crates use those features and become effectively nightly-only crates, those features become stuck in the state that they are in any way. An auto-promotion policy for features that don't keep changing but see non-trivial use would create a sense of urgency to iterate in order to avoid features getting into stable in an unideal state. It troubles me if it is viewed as a good-enough state that useful stuff is available in a de facto stable state on nightly only for extended periods of time. (I see parallels with -webkit- prefixes in CSS.)

But, yes, I don't care that much about advocating a general policy if these particular features stop acting as a gravitational pull towards nightly-only.

151

Given that impl Trait for arguments is very possible for 2017, I find the discussion on generics syntax to be on topic, but of course should be focused on adding sugar instead of breaking changes.

1 Like
152

Some features (like intrinsics) are intentionally perma-unstable. We are planning to completely change the way they are exposed in the future, intentionally breaking all nightly crates that use them. That’s why they are unstable.

153

Another summary. As before, if I’ve missed or misrepresented anything, please let me know so I can fix it ASAP.

  • Rust should provide a more seamless FFI story root 16 38 44 48 49 50 53 98 99 104 112 122 127

  • Rust should have a lower learning curve (as well as being learnable as a first language) root 4 6 9 54 63 104 112 118 81 101 119

  • Rust needs to reduce developer dependence on nightly (custom_derive, SIMD, etc.) 9 62 92 93 96 97 109 118 129 143

  • Rust needs better answers (either guides, outreach, or new features) for teams & programmers coming from OOP – this seems very related to “Rust should have a lower learning curve” to me, but could be distinct if it covers type system features for inheritance? 3 6 8 14 26 54 90 112

  • Rust needs a better story for embedded and low resource development (re: no_std, alloc failure, etc.) 9 28 45 57 76 129

  • Rust needs cleaner integration with existing tools (system toolchains, build systems, etc.) 30 33 104 105 112 122

  • Rust needs to have a more expressive type system. (type level values, CTFE, variadics, HKT) 48 52 95 99 107 128

  • Rust should be well-equipped for writing servers root 9 66 68 92 104

  • Rust should have basic IDE support root 9 63 104 118 119

  • Rust’s compiler should be significantly faster root 53 77 87 93 104

  • Rust needs to improve discoverability of community projects and initiatives which address our goals. 51 96 104 125 137

  • Rust should have 1.0-level libraries for concurrency and parallelism root 9 92 104 112

  • Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates 17 37 97 129

  • Rust needs clearer, more consistent branding. 62 104 109 106

  • Rust’s core ecosystem should be maturing, and possibly offer a more batteries-included experience. root 96 104 99

  • Rust needs to be more mature and present a more mature image to potential adopters. 104 109 104

  • Rust’s community should provide mentoring at all levels root 81 104

  • Rust needs 1.0-level libraries for scientific computing (matrix manipulations, BLAS wrappers, vector functions) – see Eigen and mshadow 52 135

  • Rust needs better error handling in production. 129 74

  • Rust needs better testing and doc generation across conditional compilation directives 16 24

  • Rust needs to encourage better interoperability between crates which occupy the same domain. 135 138

  • Rust needs to fix existing bugs and suffer fewer regressions. 64 97

  • Rust should improve its syntax for usability of common cases. 55 119

  • Rust should increase the stability and utility of its infrastructure and automation. 68 80

  • Rust’s community and processes need better visibility to “casual” participants and newcomers (e.g. RFC visibility and dashboards, casual is a bad word for this, but I can’t think of a better one here) 9 61

  • Rust’s tooling needs to be first-class to aid adoption (beyond IDE support, i.e. rustdoc) 16 104

  • Rust development should be fundable by the community. 16

  • Rust needs a clearer specification of its semantics. 112

  • Rust needs to continue focusing on what we’re doing well (performance, incremental improvements, core tools, docs, bugs & soundness issues, having a great community). 104

  • Rust needs to continue improving ergonomics. 104

  • Rust needs to define and execute better processes for improving documentation and guides across the ecosystem, but especially for core projects 21

  • Rust needs to encourage more “first contact” usage. 104

  • Rust needs to grow a new temporary team structure (strike team, accelerator, etc.) 70

  • Rust needs to grow its support for platforms other than x86 (and beyond ARM too) 28

  • Rust needs to have data-driven feature prioritization. 72

  • Rust needs to improve its distribution story for environments without internet access. 48

  • Rust needs to simplify existing features, and potentialy remove ones that introduce complexity. 67

  • Rust needs truly first-class Windows support. 112

  • Rust should track adoption metrics. 104

  • Rust’s ownership system needs to aid in modelling more complex systems (i.e. parent/child communication) 3

17 Likes
154

First: There are over 150 replies to this thread so far. I've skimmed them and nothing jumped out that I felt I was duplicating, but if I have, my apologies. Please point me to the correct existing conversations.

MOVING ON:

Ultimately, the goals listed in the original post here feel like "How does Rust compete with Go", instead of "How does rust replace C and C++". That worries me. Mostly because I'm in that vague "systems programming" space, and I want a better language; but also because I work closely with people impacted by all the easy-to-prevent security vulnerabilities in today's OSes and devices.

At first blush, the list of non-goals seems like the laundry-list of "things that would help get Rust code into places currently using C and C++":

  • Making Cargo work well in more contexts (e.g. big existing build systems)
  • Adding compiler/platform targets
  • Producing more API bindings for core platform libraries
  • Improving our story for programming embedded systems/OSes

I worry, I guess, that the direction of the project is being overly impacted by comparisons laid forth in that orange forum that many of us spend time on. There's an absolute bias there towards web-focused development and SaaS applications, which doesn't necessarily map onto Rusts strengths.

Rust, in my mind, can do the most good in embedded, OS, and other "systems" development. There are already very good languages for writing fast, correct servers. Why push Rust in that space, when it has an opportunity to stand on its own in an area where there's been no serious competition for decades?

16 Likes
155

You may have misinterpreted that heading. It was listed as "Other possibilities, or potential non-goals" -- the emphasis was supposed to be on the first part ("Other possibilities") and not the second part ("non-goals"). In other words, these are things that seem very important -- are they important enough to bump to the top of the list?

There have been several replies in this area. I'd like to discuss the needs of the "embedded" space a bit more deeply. I am thinking of opening a separate thread though for that so I'll hold off on saying much just this second.

156

So. There's a lot here. Yesterday and today I sat down and re-read through all the comments on this thread. Let me say up front: @dikaiosune, your summary comments are invaluable, thank you so much for that!

One thing I noticed is that a lot of the ideas here already fall under some of the existing priorities (which is great). In many cases you can see that from the summary that @dikaiosune wrote, where he linked things to [root]. I think there are also a lot of cases (e.g., "Rust needs to continue improving ergonomics") that he didn't link to [root] but which (to my mind) are already covered (e.g., under "improve learning curve").

Digging deeper into some areas

In terms of moving the discussion forward, I wanted to try a thing. I branched off a thread to dig deeper into the needs for embedded development. I would like to see threads branched off for a few other areas that I think merit some deeper attention:

  • Productivity: learning curve and expressiveness
    • I think we all agree this is important, but I've been wanting to start some threads trying to expound on the right ways to do this, and in particular with a focus on "positive sum" outcomes where we don't pit new vs experienced users against one another
  • HPC / numerical computing.
  • Seamless FFI.
  • Game development, perhaps? I didn't see a lot of citations here.
  • Windows development needs (I hear mixed stories here?)
  • Dashboard and visibility
    • visibility into the state of things has been very much on our minds, and part of the point of the roadmap process was to help lay the framing for that
    • @dikaiosune and @brson have been doing great work here with http://rusty-dash.com/
  • First contact: @nrc mentioned ways to make it easier to experiment with Rust and cited React/Angular, I'd like to hear more about what he has in mind

I am going to try and fork off some of these threads, but I may not get to all of them today. Feel free to start up a thread yourself. I think the main thing is that it would be really helpful if we can focus on the most burning problems and also on categorizing the space. A lot of domains (cough HPC, embedded cough) are really enormous and varied. What is a blocking requirement in one case may be a non-issue in others, and it's helpful to clarify some of these distinctions for us noobs.

One thing I want to say: I feel like these threads are useful irrespective of whether we add some text about that particular topic to the roadmap. It's awesome to dig into these areas in depth.

Possible non-goals

Now, one thing I also want to bring up are non-goals. I did see a number of topics raised that I think don't rise to the level of "Rust project goals". There are a couple of different reasons for this:

  • Often, the idea itself may be sound, but it is presented without motivation. I think we really want to focus on the problem we are trying to solve.
  • Sometimes, the idea doesn't seem to require "community focus", but might better be pursued independently.
  • Finally, some things just don't seem as pressing as others.

Here are some candidates that I found. I'll just quote from @dikaisune's summary:

Rust needs to have a more expressive type system. (type level values, CTFE, variadics, HKT)

This is a classic case where I think we need more motivation and less mechanism. I don't want (e.g.) HKT for HKT's sake. I want to know what we plan to do with HKT (which may well affect the precise design of it). Basically I think these kinds of language extensions don't belong on the roadmap, but the problems we aim to solve with them may. Anyway, my take on some popularly floated proposals:

  • const generics and CTFE seem like a valuable avenue for unlocking many high-performance applications (e.g., HPC, small vec, embedded spaces); we should aim for something minimal. @eddyb has already been pursuing a lot of the important refactoring as well of course work by @scott and others on miri.
  • HKT: the application heres are potentially widespread, e.g., Iterable traits, being generic over Rc vs Arc, but HKT as found in Haskell may not be the best solution (and raises a lot of open questions). I like RFC 1598 by @withoutboats quite a lot. It's not HKT, but it's a relatively small, natural extension to the language that can express HKT, and would address things like the Iterable trait and so forth. Not sure how to prioritize this: digging more into the applications may help!
  • variadics: the primary motivation to pursue variadics for me has been making it possible to implement the Fn traits. I think we can do this in other ways (e.g., adding syntax like impl Fn() for T) that would let us punt this a little further down the road. Given that I would want said syntax anyway, I'm sort of inclined to do that. =) (But maybe a good variadic solution is not so much work, not sure). In any case, I'm not sure just where this falls on the roadmap tbh, but I could imagine more power vis a vis Fn traits affecting a lot of areas a little bit (e.g., @carllerche raises this point to me regularly).

Rust needs to have data-driven feature prioritization.

The basic idea here is polling for what features people need etc. I am very much in favor of trying to get numerical data on what people are using and also trying to find out what people want. But I would not want to drive our decisions based on a purely numerical basis. For one thing, good polling is hard (citation: world events) -- as soon as you turn "polls" into (effectively) "votes", you raise incentives for people to do things like twitter-based vote drivers for their pet feature and so forth. I would rather that we try to evaluate things on the merits. But I do think that broad-based surveys are really useful for bringing things to awareness that were flying under the radar.

Rust needs a clearer specification of its semantics.

I think that as part of servicing other goals -- e.g., mature ecosystem and tooling -- we should focus on specifying the areas most urgently needed: unsafe code guidelines and grammar. Trying to formalize and specify all aspects of the type system seems like a big endeavor with very little payoff. It doesn't seem to rise to the level of a roadmap priority for me.

That said, I am very interested in making it easier to be more formal in RFCs. We've done some academic collaborations (e.g., this formalization of the trait system done at UBC and of course the ongoing Rust Belt project), but these efforts tend to lag and subset the language. What is needed to help with RFCs is something much more complete. I think improving and simplifying our trait implementation would be a great starting point. I've been toying with ideas here and would love to dig into them (ah so many half-written blog posts).

Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates

This seems important, but I don't know that it rises to the level of a roadmap point. Basically, if we find a good way to do it, I think we should just do it. But is hashing what's really preventing Rust from making in-roads? (I guess maybe?)

5 Likes
157

Oh, things I learned recently: If you want to start a new thread based on an existing post, you can click on the "age indicator" (the same way you would get a permalink) and click on "+ New Topic". This gives better metadata to discourse (which of course maybe we can employ for rustydash someday....). Know your tools! :smile:

1 Like
158

Plugins are definitely a life changer, but other than what's already been said I don't see anything to add.

159

I agree. Modelling DAGs in Rust is definitely not fun.

And lexical lifetimes…

160

Moreover, the dispatching can be made easier using my ifunky crate. A higher-level crate could combine it with a cpuid crate to do something closer to GCC function multiversioning (as opposed to bare ifuncs), though without the nice "automagically use the additional CPU features during optimization without requiring separate compilation."

161

An RFC based on this discussion has now been opened!

4 Likes
162

One thing we do miss to really pitch Rust in our workplace is a good coverage story.

Right now it’s a wonky experience, and that’s something we’d need to push forward rust components in our stack (distributed servers, one of the main focus if I trust the RFC). I remember stating the same during the Rust survey, but since it wasn’t a default choice, there’s a big chance many people didn’t think about it.

We’re a linux shop so we’re not worried about lacking windows or OSX support, but I kind of get the feeling such a feature wouldn’t see the light of the day without a full support from tier1 platforms. Am I mistaken? I’d be willing to spend time getting that right for the *nix part.