Setting our vision for the 2017 cycle

I should’ve been explicit about this. The order does roughly reflect the sense one gets from the survey results, but it’s rough at best. I’m not sure whether a ranking of the final vision statements is a good idea, vs trying to scope them so that it’s feasible to get all of their goals done in a year.

I cannot use Rust in production because its abstraction capabilities are not powerful enough to tackle the problems that arise in my domain.

My application domain is, ironically, high performance computing (numerical simulations). The main piece missing is type level values, but there are also others.

I need type level values to abstract algorithms and data structures over the spatial dimensions of whatever problem I am trying to solve, such that I only have to implement them once. Quad/Octrees, BVHs, structured/unstructured meshes of simplices or hexahedras generalize to arbitrary spatial dimensions. Most numerical methods for PDEs (finite differences/volumes/elements, DG, Lattice Boltzmann) do so as well.

Type level values also would allow better fundamental libraries for, e.g., linear algebra.

But with linear algebra we come to the second piece missing: better support for EDSLs. EDSLs make libraries easy to use, but they are extremely hard to implement in Rust (harder than in C++ and not as powerful). Anything that improves Rust EDSL story would help us implement the libraries that we need. From the top of my head, procedural macros, (way) better CTFE, variadics, and maybe HKTs would help.

There are also some issues with FFI integration. HPC code relies heavily on MPI, whose standard define some macros, that might have different types depending on the implementation used. This makes writing a rust wrapper around MPI a bit of a hassle because those macros have to be dealt with at the build system level, instead of within Rust itself.

Finally, most HPC environments don’t have direct access to the internet. This means no rustup.rs, no cargo, no nothing. It would be great if there were ways to use these tools from remote systems, e.g., via ssh tunnels, or if there was an easy way to prepare the environment for a crate in some system, and ship it easily to another one (where everything gets recompiled).

So while I would love to use Rust in production for high performance work, the language currently doesn’t allows us to abstract at the level that we need to do our job efficiently. IDE integration, better compile times, lower learning curve… all of these are great, but even if all of these are fixed we would still be unable to use Rust in production.

For this reason I would like to see in the vision for 2017 how Rust plans to expand to those system level/low-level programming fields that it currently struggles at. My field cannot be the only one, so it would be nice to hear of others as well.

EDIT: I rephrased the post because its 1:30am here and the points I tried to make were even unclearer than they are now. EDIT 2: I think it is actually sad, that Rust concurrency is so good for its age, and that we have so many massively parallel problems (as well as access to so many cores), but cannot use it :frowning:

15 Likes

I’m glad to see FFI included in these goals, and in particular the mention of seamless #include support. In particular, that gives me hope for detecting library version issues at compile time rather than via runtime crashes. Right now, libfoo-sys crates declare the various C symbols they use, with no way of knowing if those declarations match those of the version of the library the crate compiles against.

3 Likes

Like @Eh2406 said - the rust-cpython bindings are actually really solid. I’m working on a blog post ATM about how to write extension modules for Python in Rust. Still, I agree that the story there can be improved, especially if the work on custom garbage collectors makes progress and the interop with Python’s refcounting can be worked out. I’ll need to take a look at how that can work with Cython too.

3 Likes

This reminds me: one of our perpetual problems is just making good stuff that exists more visible. We’ve added a lot of links within the web site, but it ends up being totally overwhelming.

How can we make it easier for people to discover things like Rust’s Python story?

One thing that I discussed with @aturon a while ago would be building the fundamentals for a scientific Rust stack. It’s not a lot of effort to get the initial core fundamentals out, but once it’s there it would be a central place that other disparate projects could build upon. Think NumPy ndarrays for Python, the Julia NDArray, a subset of Eigen, etc.

The core primitives could be:

  • a core ndarray abstraction
  • BLAS wrappers on top of these
  • scientific functions on these arrays (e.g. the vector operations in MKL/Accelerate/NumPy)

Some inspirations from the C++ side (that are probably overly generic but are reasonable) are Eigen and mshadow.

With these, the scientific Rust ecosystem would have a central place to build upon.

8 Likes

This is the most important thing for me. There are things which I am currently incapable of binding properly in winapi due to a lack of FFI features. Things like unions, #[repr(align(N))], #[repr(packed(N))], bitfields, COM calling conventions, dlimport, static-nobundle, unsized types with thin pointers, and more. In fact, I don’t think Rust has had a single FFI feature implemented since 1.0 (the most useful thing for me that did happen since 1.0 was making glob imports actually work). Several of those features I listed even have accepted RFCs, so I have hope, but seeing them take so long to be implemented, nevermind stabilized, is really demoralizing.

This is also hugely important, not just for me but for everyone. I’m going to some fairly extreme lengths in winapi just to minimize the amount of code that has to be compiled so compile times can be fast: getting rid of almost all trait impls, making enums simple integer constants, and abusing cargo features. I only have about 10% of the code currently compiling in winapi 0.3 but it already takes 4 seconds to build. So, when I do eventually get everything updated, should I look forward to 40 second compile times?

7 Likes

I think it would nice to have a few thorough guides that cover features that many older mainstream languages have but rust doesn’t.

Just a few things that come to mind:

  • Working with generics.
  • How todo OOP in rust.
  • Working with Iterators.
  • How to do error handling without exceptions.

One thing we’ve talked about quite a bit but never quite gets off the ground is a single “derive” for common data-types.

Straw-man:

#[derive(Data)]
// equivalent to
#[derive(Copy | Clone, Eq, PartialEq, Ord, PartialOrd, Hash, Debug)
// what about Default and Zero? Should Data include them if possible?
4 Likes

This is a beautiful example of the kind of curve-bending that the Rust community has gotten so good at. Instead of just assuming that some ostensible tradeoff is a hard tradeoff, identify ways to get many of the benefits of both solutions at the same time.

That shrinks down the set of cases that aren’t reasonably handled by the default (in this case, codebases with both fast-hash requirements and significant code size limitations), which makes the justification for escape valves much easier for users to understand.

One thing that could be improved with #[no_std] is getting some of the core crates (like alloc) stabilized.

I’m a little confused about that one in particular – are we likely to be changing alloc::heap::allocate in the future? It seems pretty basic to me.

1 Like

I don’t know a lot about this, but I’m curious why you would not want to use cross-compilation, e.g. with something like a MUSL target that has basically zero dependencies.

It feels like we have a body of data now – it’d be good to know what sets of traits are commonly implemented together. I feel like one of the problems with a shorthand like that is that I fear everybody has a slightly different subset in mind.

Doesn’t shoggoth.rs at least somewhat cover this for now? (I admit, native support would be nicer).

1 Like

:+1:

For 2017 I would like to see two things.

  1. Serde json should work out of the box on stable release channels with no customizations to my build for code gen. This has been a sour spot in the past year as the community transitioned off of rust serialize to serde where rust serialize worked out of the box and serde required build assembly. Effortless Json support is kind of a bid deal when it comes to network services. We should make rust more seemless to get up and running with a good json lib on the stable channel.

  2. more ferris. Or to somehow make ferris more official. It’s weird there’s an unofficial mascot and an official logo. I’ve personally seen this cause confusion. Rust should have a consistent branding. It deserves it! 2017 could be the year of ferris. Let’s make it happen.

7 Likes

I was one of those people who bounced off Rust. It has a lot of things I like, so I keep checking back to see if it has what I need yet. I’m glad you’re looking into usability and learning curve. We evaluated Rust vs Go at my last company, and a couple of us found that after 2 days learning Go we were productive, and after 2 days learning Rust we were fighting it. I feel like I need a few things: better error messages, an IDE that can tell me “what type is this expression”. The hardest thing was the type system and especially around the reference checking types. I got stuck between the difference of “what type needs to go here” and “what type is the expression I just wrote”. Error messages got better lately, more of that please! I hope the IDE support comes along soon too. (I like Emacs, but could see switching to Sublime). Thanks! I hope Rust keeps growing.

2 Likes

A fix for type inference regressions. It should be possible for crates to record fully qualified function names somewhere and use these when type inference fails.

  1. All packaged crates (e.g., on crates.io) should include this information so that rust upgrades never break type inference for dependencies.
  2. Rust could also store this information in the target directory and provide an option to “fix” one’s code on upgrade. Alternatively, it could store it in some file analogous to Cargo.lock. That way, it could be checked into version control so that anyone downloading the code can benefit from it.

IMO, this would cover most cases. However, crater should probably turn this feature off so we can avoid introducing type inference regressions in the first place when possible.

This isn’t quite as straightforward as it appears as plugins aren’t deterministic however, most plugins use fully qualified function names anyways.

4 Likes

I think improve rust-lang self should be the highest priority task in a year or two, [Zeroing in: closing gaps in our key features] (https://blog.rust-lang.org/2015/08/14/Next-year.html).

As as rust beginner, I am most concerned about is the [Non-lexical lifetime] (https://github.com/rust-lang/rfcs/issues/811) and Plugins. I thinks rust’s lifetime system all are naturally (of course not easy) except lexical lifetime checker,but this key feature appears to be little progress. for compiler plugins,thanks Procedural macros 1.1, but Procedural macros 2.0 also seems couldn’t finish in a short time.

5 Likes

I don’t particularly want any language features. They’re nice to have, but I feel that the gain from adding new language features at this point is pretty marginal. You can see that in this thread: everyone really, really wants one different language feature. If we work on some language feature that one in 10 people wants, we disappoint the other 9 in 10 people.

In particular, I think that the importance of non-lexical lifetimes shouldn’t be overstated. It’s something I’d like for sure, but I’d caution against “we can compete with Go [or substitute any other GC’d language] if only we had non-lexical lifetimes” reasoning. We will never free the programmer from having to learn about lifetimes. We will never have as low a learning curve as Go/etc. in the memory management department. Competing there should be a non-goal. (That said, it’s absolutely worth making lifetimes easier to use, of course.)

Compiler improvements are welcomed; I’d love to see incremental compilation and MIR improvements.

By far, though, the most important thing to me is having a good story around servers. Right now, when you want to write a network-facing app in Rust, you have a myriad of choices, each of which with significant drawbacks. I’d like to see the futures/async I/O stuff finished, which would solve this problem in an elegant and Rusty way.

A meta-note: I’d like to humbly request that people refrain from further “I’m disappointed that X isn’t implemented yet” comments. They don’t move the discussion in any productive direction. Consider that “I’m demoralized” comments apply not only to you as a user but also to us as compiler developers. Let’s focus on the concrete use cases.

29 Likes