[Roadmap 2017] Productivity: learning curve and expressiveness


#30

Re the string problem. I think a nice solution would be to add support for ‘from literal’. Then we can have a SimpleString crate which has a string type that newcomers can use without worrying about the more complex string types and which allocates all over the place. Opting out of the allocation means just not using that crate. Some support for creating a SimpleString (or Str or whatever) from a string literal token would be needed to make this transparent.

Support for custom literals would also be useful for BigNum types and other numerical libraries.

In terms of implementation, the first thing to come to mind is to allow crates to provide the functionality as a procedural macro (tagged with an attribute for identification), that is then run normally (though implicitly) in macro expansion. The downside of this is time to stabilisation, but since the macro needs would be very simple, we could fast track it in the same way as custom derive.


#31

Looking at the big picture/requirements rather than solutions, etc. I think it is great to push on this stuff, I’m especially keen to see us tackle some of the smaller, low-hanging fruit solutions rather than the sweeping changes. I think focussing on ergonomic paper cuts is way to get big returns for relatively little effort and with relatively few risks to the language. In contrast, I worry deeply about adding to our complexity budget with big new features.

The fact that string literals have type &'static str which cannot be readily coerced to String

ref and ref mut on match bindings are confusing and annoying

lexical lifetimes on borrows

Fiddling around with integer sizes

These all seem like pure win to me. NLL are a big issue, I don’t think it significantly affects the complexity of the language (for users, clearly it complicates the implementation). However, I think we have significant non-technical debt in terms of documentation and tooling for lifetimes/borrowing and I think this feature adds to that debt. We should prioritise paying some of it off too.

References to copy types (e.g., &u32), references on comparison operations

I think this a worthy target, but whether it can be addressed depends on the solution. If there are nice solutions it seems like something good to do.

Some kind of auto-clone

This I’m not keen to think of as a goal. If we want to tackle the higher-level space, then I think we need to think hard about an holistic strategy rather than polishing specific pain points that might be detrimental elsewhere. Put another way, although it is annoying, I don’t think cloning Rc/Arc is a pain point for most Rust code today.

By strategy, I mean that maybe for higher-level programming we want a new type like Gc which didn’t need cloning or even a new language (RustScript!) or language mode. I.e., there may be bigger language changes that give the desired ergonomics here.

#[derive(PartialEq, Eq, PartialOrd, Ord, Copy, Clone, etc)]

derive shorthands seem like an obvious and low cost thing to add, I’d love to see this.

language-level support for AsRef-like pattern

Could you expand on why you think this needs language-level support? My experience has been that (mostly) it works smoothly as is. The risk/reward on this one seems much higher than the others.

lifting lifetime/type parameters to modules

I would like to see this, but it does seem to add to the complexity budget a bit. I’d be happy to put this one off for a while longer.

inferring T: 'x annotations at the type level

Would be great to play around with elision/inference here.

explicit types on statics and constants

I think we could cautiously add more inference here, perhaps starting by bailing if we need cross-item dependencies?

trait aliases

Trait aliases (as in something like type but for traits) seem quite different from inferred bounds, why do you think they are related? I’m very keen on trait aliases, not sure about inferring bounds

We’ve also discussed type and lifetime parameters on impls as a paper-cut we might tackle.

I’ve warmed up to the idea of getting rid of extern crate, although I’m still not very convinced it would buy us a lot.

Seems worth thinking about if we can simplify the module rules too.

I would also like to tackle some of the boilerplate around structs (derive(new), default values for fields), and add enum variant types (currently blocked on finalising the default generics stuff).

I want enough of CTFE stabilised to have RefCells in statics.

#[transparent] (#1744) for easing the ugly stack traces we give out.

On the bigger items, I strongly think we should focus on finishing off the stuff we have in flight, rather than adding to that mountain. It would be much better for the language to finish and stabilise specialisation, impl trait, default generics, CTFE, allocators, etc. than to start on const generics, virtual structs, or HKT-ish stuff.


#32

another ergonomics nit (and related to virtual structs): I would like to have some solution to the ‘newtype deriving’ problem, i.e., some way to write an opaque type with no access to the internals (i.e., like a newtype, unlike a type alias), but that allows access to all public (or accessible) fields/methods without using Deref (which is inappropriate and has problems with priv type in pub signature errors) and without writing a bunch of boilerplate impls.


#33

The recent trend of trying to redirect Rust from a fast, safe systems programming language to a language focusing on developer productivity is worrisome. One example is the requests to reduce large portion of explicitness in language in order to either type fewer key-strokes, or to avoid thinking how work is done under-hood. It’s fine if some one who does not care about low-level detail finds a way to write fast code, or a Rust beginner to trip less on compiler, but if this means compiler becomes easier to accept inefficient program, or it’s harder for programmers to spot previously easily noticeable bugs, then we should have a second thought. Rust is “the” system language that a lot of people have been hoping for for a long time, which has the potential to fundamentally change the landscape of critical infrastructures of this information age. No second language in sight has the ability to do this. In the mean time, while it is nice to have yet another high productivity language, this need is arguably less urgent in an era when multiple high quality successful dynamic-typed or garbage-collected languages already exist.


#34

I absolutely agree, and feel the same way. That said, to change sides for a moment: the more people using Rust, the stronger the network effect can be. More community resources, more publicity, more available programmers, more jobs… all of those would possibly be helped by helping more people use Rust. And the more people using Rust, the fewer projects that may need to turn to unsafer languages for performance, or slower languages for safety.

Plus, I also believe there’s a point of diminishing returns on explicitness: at some point, you start skimming or cutting corners because it’s all just too much to deal with.

It’s a tightrope walking act, really. I think we need people arguing both sides.


#35

Modern languages have shown that you can design a safe system language that’s also sufficiently handy to use and sufficiently predictable. This also allows lot of people to avoid switching to a more handy language, you can use Rust for more purposes.

Your concerns are reasonable, but they should be applied only on each specific feature we want to change or add, and not as a blanket on the whole efforts of trying to make Rust more handy.


#36

I’m 100% sure I can speak for the rest of the language team, and probably the core team too, that we have no intention at all of making Rust any less of a “fast, safe systems programming language”. While we do want to focus on developer productivity, we will only do so where we can make programming in Rust more productive as well as keeping it performant and safe. We strongly believe you can have all three.

Of course opinions may differ about exactly how we keep being fast and safe at the same time as improving productivity, and that is why we have these discussions in public, to make sure we are doing the right thing.


#37

We don’t have any particular feature proposal to discuss the trade offs of, so I think a lot of the responses on this thread are premature. Maybe what you think you don’t like about an idea in Niko’s post isn’t actually true of the proposal he has in mind. We can’t talk about the performance implications (for example) until we have actual RFCs.

I think the thesis question of this thread is this:

  • Is Rust as accessible and easy to use as we want it to be? If not, is it worth putting focus on solving that problem this year?

#38

All things considered, what I want most is that Rust not add more implicit features or coersions. I could say (or rant) more, but perhaps it’s best to just leave it at this: implicit/coersion == bad.


#39

I’m interested in what people mean by productivity? Given that I feel most productive in C++, I don’t quite understand what other people want or how many requested features even achieve this.

Interestingly, you didn’t mention what was said about Rust vs Go in terms of learning curve, etc:

However, this does show that Rust’s infamous learning curve isn’t that bad, especially compared to the much touted simplicity of Go. Furthermore, becoming familiar with a language is basically a one time cost. I wouldn’t be surprised to find that if this experiment were done by someone who is expert in Go and Rust instead of a beginner, the Rust version would be completed faster than the Go version.

Initially, I thought having a #[derive(ValueType)] would be nice too, until I realized that perhapsValueTypemay not end up being what I want it to mean. And then I started thinking about the two usability scenarios forderive`, which is writing and reading.

When writing a derive, in almost all cases, I need to find the docs (this should be easier, since it’s not in either reference (libs or language) and instead, is only in the book) for what things are supported and then I need to determine whether my type needs a custom implementation or whether the derive one works. At this point, I fail to see how not having to type out each one results in a win. The only scenario that results in a time saving is that you blindly type #[derive(ValueType)] and it works without complaint and you don’t have to think about it anymore. But I personally wouldn’t want to optimize for that.

When reading, however, I’d much rather have a list of traits that are derived rather than having to read the documentation to figure out whether I remember exactly which incarnation of ValueType’s Rust decided to use.

So, what’s the value of a shorthand when it complicates reading and doesn’t significantly change writing?


#40

This. As for writing, this is a problem which should be solved by tools. I want to type out,

struct Foo {
    a: String,
    b: i32,
}

have a light bulb appear near this struct, hit Alt+Enter and get derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug) above the struct. This is not quite possible yet, because IntelliJ Rust does not know yet if it is possible to derive a trait for the struct (will try to implement this), and the idea of such intentions and quick fixes is not present in RLS/racer (please correct me if I am wrong, I have not double checked this). Though even simple syntactic based assistance can make writing derive more pleasant: intention to add the #derive() itself + completion for common traits (a gif of similar feature in Intellij Rust) + completion of pairs of traits (TODO).

This is just an example though. The main theme is that it is the tooling and not the language itself that defines productivity. Java is a very verbose language, but you can be extremely productive in it if you use Eclipse or IDEA. And you don’t even need to be an expert, because these tools, unlike the language, are automatically descoverable.

To be clear, Rust is concise and dense, and this is great, but adding alternative ways to do something just because the proper tooling is nonexistent at the moment is a bad idea imo.

To bored to type all the derives? Let the tool type for you, because it knows which traits are automatically derivable.

Have “Hello, world”, but want String? That’s easy, it is a compiler error, so hitting Alt+Enter should just add an into() call.

Accidentally moved out of a struct with a pattern? That’s easy too, Alt+Enter and here is your ref. And if you want to get fancy, add an intention to switch all the patterns in the match between destructing moves and inspecting refs.

There is &&i32 in your chain of iterators? Warn the users if they match it as x and not as &&x in the lambda argument, and suggest as quick fix.

The problem of productivity and learning curve is huge, but imo the solution is not more language design, but better tools support (and tools are not about completion. Hippie expand works great in practice! Tools are about helping to look at your program as a Rust program, and not as a sequence of Unicode characters).

Looks like I’ve got carried away a bit and even wrote a post without a smiley :fearful:


#41

Why is it a problem that should be solved by tools?

Sure, it’s a problem that can be solved with tools, like a wide variety of language usability issues. As you say, Java can be quite productive despite its verbosity. But a lot of people also hate Java for that verbosity (including me), because, among other things, (a) code is read more than it’s written, and it’s harder to determine what a piece of code does or whether it’s correct if it’s full of useless fluff, and (b) not being able to sanely write code without an IDE is annoying.

You could just as well say that (like Java) Rust should not have type inference because you can just have the IDE complete all your types.

I do not mean to imply that I don’t think tooling is ever the answer. When explicitness makes code easier to read rather than harder, I agree tooling can be a good approach to help programmers deal with the resulting cognitive burden (as long as it’s not effectively mandatory). Indeed, rustc on the command line already provides suggestions, although it is not so nicely integrated as an IDE. But it’s hard to argue that a spew of derived trait names passes that test.


#42

Agree, my statement of “the problem of productivity and learning curve should be solved by tools” is wrong, because this problem actually consists of a ton of subproblems, and only a part of those problems could be solved by tools.

But I stand by the

adding alternative ways to do something just because the proper tooling is nonexistent at the moment is a bad idea.

statement. I find it disturbing that for a lot of small cases a language solution is proposed, and the tooling solution is not even considered. This is understandable, because changing a language is easy, there is a compiler which you can just hack on, while providing the tooling is very difficult, because the basic infrastructure for inspecting and editing rust code is nonexistent.

As an example, let’s consider this derive problem again. Here is my simple solution: just add an action, which is active when you are inside a struct definition with “value” fields, which adds all necessary “value” derives. Can you write a hundred lines of code to implement it? Looks like the answer is no, because where should you add those lines? To the compiler? But it knows nothing about editing the code, and it is not interactive. To your favorite editor? Well, there are 2 + many editors out there, and none of them is capable of doing this properly, because they know nothing about syntax and semantics of Rust.

You can’t implement a simple (hundred of lines if you have the necessary infrastructure) tooling solution, so you necessary need to push for a language solution. And this is a problem which is worth solving imo.

I agree that tooling is not always the solution (though I think that at least for 4 problems I have described in the post the tooling is the best solution), but the current problem with Rust is that it is never a solution.

EDIT:

And, for example, to solve derive problem in Dart, you would need to write those 100 lines here, and it would be available in any editor.


#43

I’m writing this on my phone, so I’m sorry for any and all overlooked spelling and grammar mistakes.

On the topic of solving by using “tools”, I’ll agree wholeheartedly if we consider macros tools. As for explicitly relying on a set of external IDEs, I’m not sure I think that’s the right solution for many of these issues.

This! As someone who spent many of my early months in programming Racket I can’t even begin to express how well the family of language concept works. This is why I was pushing on the Macro 2.0 to avoid ! in the names, since it would allow macros to become a tool for sub-language development. I personally find myself wanting Rust-like types without Rust-like ownership and static allocation. Rustscript (for lack of a better name) would be a killer feature. This isn’t even really a new concept in Rust, as we already have unsafe.

fn foo() {
    script {
        // Could `new` be a macro?
        let x = new Foo;
        let y = x;
        println!("{:?} {:?}", x, y);
    }
}

In response to the OP though:

I personally never find this all that hard to deal with, but I don’t spend a lot of time interoperating with C libraries in Rust at the moment. I like that my base numeric types feel very un-magical.

Figuring out what language support is needed from Rust to have a nice object system (might be using the wrong terms here) is key. Every project I’ve written with a complex data model has eventually needed something like this in places. Bridging the gap between the relational mental model and the functional immutable model could be really interesting in Rust. My only real fear is that doing this in a way that upholds the zero-cost or very clear cost is going to be hard.

Yes please.

Interesting. I actually really like this idea, the general type parameters often give you a high level sense of the kinds of places you might want to use something, having this show up once at the module level seems like a readability win in a big way.

Kinda related but, I always find myself needing to explain the orphan rule carefully, and the reasoning isn’t completely clear at first to new users.

I think that concepts from Design by Contract should be considered as a part of this again. Racket for example has a really good story for runtime error messages. I would be sad if Rust aims for anything less.

Two kinds of programming language I want; Magical, and Non-magical. Rust is very clearly in the non-magical category, or should be.

I also want to add that despite that the module system is one of my favorite things about Rust, it can be a bit of a pain point in the learning process. I’m not about to propose something as crazy as automatic name resolution or something along those lines, but I can definitely see the concept of re-exporting types for the purposes of modularity and privacy being new to people.


#44

My point was that in all cases, I want to see an explicit list of the traits being derived, in the #[derive()] macro. I.e. I don’t want to see an aggregate or shorthand, because I find it less readable. The focus here being on what text I want to see in the source file, rather than how that text was generated. Some verbosity is bad and some is good.


#45

I disagree about macros, because macros in a lot of cases are crutches for missing language features. println! is a good macro, because it does something that’s non-trivial. A lot of macro usage in Rust is like “Rust doesn’t support type level integers, so just macro it up to the 10 case and wait for the new feature” or “it’s awkward or impossible to write this kind of polymorphic code in Rust, so just write a macro and pass types into it”

In other words, if your macro is doing the work of a C++ template, that means the current generics implementation is not working for you.


#46

Another example, two macros from itertools:

for (i, j, k) in iproduct!(0..4, 0..4, 0..4) {
for (i, a, b) in izip!(0..100, &mut xs, &ys) {

You can’t add everything to a language, so having a meta-feature that allows you to build very specialized features is probably handy.

On the other hand, variadic typesafe generic functions as implemented in D language are useful in sufficiently common situations.

There are reasonable persons that think that C++ templates are used for too many things. You can argue that more specialized and more typesafe generics are better than the “dynamically typed” (at compile time) templates of C++.


#47

That’s what I’m saying, macros are not as safe as generics. There should be as little as possible macros and as much as possible generic code. This is because generic code can be type-checked at declaration and give nice error messages.


#48

Right, but keep in mind that if you want to implement generics (and variadics) as powerful as in D language while keeping type safety (D generics are not type safe at compile-time), you have to introduce several higher level type system features that today you find only in Haskell and few dependently typed languages (I am not sure even Scala is powerful enough). Perhaps Rust will gain such features someday, but it’s a big complexity budget. In the meantime Rust macros allow you to do something similar in a less type safe way with quite smaller type system and language complexity (and usage difficulty, with much less “type-astronautics”). As it often happens it’s an engineering trade-off.


#49

I think that is an example of people using macros to get around a problem of the laungauge at the moment. That seems like a really really good thing to me. Can you imagine how much more irritating things would be without macros.

EDIT: How are macros less safe?