[Roadmap 2017] Productivity: learning curve and expressiveness

[leonardo]

For deref coercion, I remember my own confusion very concretely. It wasn’t so much deref coercion itself, but that there are all kinds of convenience impls in the standard library like all these by reference impls of Add or how many traits are implemented for references to types that implement that trait.

I agree, that was confusing. In some situations you need to dereference and other cases you don’t need to do it. In C language this doesn’t happen, the semantics is much more stable. This requires more precision to write the correct code, but the machine semantics you build in your head is a bit more clean and simple. Overall I prefer the current Rust situation, but don’t simple consistent semantics is important for newbies.

[leonardo]

Before we added deref coercions, you would have had to write foo(&**x). Clearly this is more explicit, but is it adding any value?

A simple example:

fn foo(v2: &mut [u32]) {}

fn main() {
    let mut v1 = vec![0u32; 5];
    foo(&v1);
}


error[E0308]: mismatched types
 --> ...\test.rs:6:9
  |
6 |     foo(&v1);
  |         ^^^ types differ in mutability
  |
  = note: expected type `&mut [u32]`
  = note:    found type `&std::vec::Vec<u32>`

error: aborting due to previous error

The error message is correct, but it’s saying two incompatible things: it says that types differ in mutability, but that’s not true, because one is a mutable slice and the other is a Vec reference, and you can see if from the two bottom lines, that show the first error message is not really true. I think this whole error message is acceptable, but it shows that automatic coercion makes error messages less easy for the compiler to give, and less easy for the programmer to understand.

[nikomatsakis]

I think this whole error message is acceptable, but it shows that automatic coercion makes error messages less easy for the compiler to give, and less easy for the programmer to understand.

This brings up a good point. Error messages can often be a decisive factor. This one is an interesting case. It is actually completely correct: the problem is exactly the mutability, and only the mutability. That is, if one wrote &mut v1, the code would work fine. But it’s kind of getting it right by accident, since it is only looking at the outermost layers of the type (it sees &mut _ vs & _). This makes it not as clear as it could otherwise be, perhaps.

In general, I think that another tool we can use to help people understand better is to really scrutinize our errors in common situations and look at them as “teachable moments”. It is a lot of work though!

(There are also some proposed languages changes that I think live-or-die on the error messages. That is, a change can easily be if we do the work to make good error messages, and otherwise will wind up being very confusing.)

[petrochenkov]

I think about magic convenience features mostly from explaining/teaching (not necessarily to beginners)/specification point of view. Basically this: https://www.youtube.com/watch?v=48kP_Ssg2eY (sorry, it’s a long talk, but incredibly useful).

Is the language rule can be clearly described in a few sentences? Does a programmer understand what they do when they write some piece of code? Are any traps and caveats waiting them after making a small step from the well beaten path? If it can’t and they don’t then the feature is evil and introducing it is a drawback that needs to be overcome by really strong motivation.

Deref coercions are not so evil - they are not MAGIC, there’s a simple scheme. Autoref/autoderef are evil, closure inference is very evil, full type inference, especially with all its accumulated special cases and patches, is evil beyond recognition, good luck specifying it and explaining why something compiles and something not, or creating an alternative compiler implementation. However, the language without autoref/deref would be completely unusable, so we have to make a sacrifice and introduce this irregular piece of magic. Or remember .as_slice()? Closure inference is much more arguable (I personally like the C++ scheme better). And lack of ref inference in patterns? Not even close to making the language unusable.

---

Implicit widening conversions are another story - they are easy to explain, but it’s probably my professional bias makes me dislike them, I mostly write hardware emulation code these days and it deals with fixed-sized integers and conversions between them a lot, but they’d better be explicit, because implicitly assigning some 8-bit thingy to a whole 32-bit thing is not desirable at all.

---

Regarding ergonomics improvements, I’d like more effort to be put, for example, into better non-exhaustive pattern matching. I’d want to write

if my_x is Some(x) && x > 10 && x < 20 {
    println!("{}", x);
}

instead of nested if lets and ifs. (And to completely eliminate if let eventually in favor of something composable, expression-oriented and naturally ordered).

[nielsle]

It would be nice to have an IDE-tool for explaining types of variables. I would like to point at a variable in the IDE and ask: Why is this variable a reference? Why is it not mutable/Copy/Clone/Sync/Display? What could I do to fix it?

Coercions, Iterator traits and similar sugar is great for productivity, but is also makes the language feel slightly magical. Perhaps an interactive IDE could help solve this.

[glaebhoerl]

I’d want to write

if my_x is Some(x) && x > 10 && x < 20 { println!("{}", x); }

instead of nested if lets and ifs. [/quote]

Fwiw if the if guards on match arms were supported with if let as well, you could write:

if let Some(x) = my_x if x > 10 && x < 20 { ... }

Furthermore, we could extend if guards on patterns to support the if let form as well; then we could chain ifs and if lets arbitrarily. (This is the same feature as Haskell’s pattern guards, just more syntactically consistent.)

[quote](And to completely eliminate if let eventually in favor of something composable, expression-oriented and naturally ordered).

What are you thinking of w.r.t. these properties?

[djc]

This brings up a good point. Error messages can often be a decisive factor.

Riffing on this a little, further improvements to error messages could also help a great deal with learning curves. For example, I found this practice of having “erased” types in error messages (as you mention, &mut _ vs & _) very hard to read. IMO the error message would be much easier to process if it stated the full expected type vs the full encountered type.

[withoutboats]

I could be wrong, but I believe the only time an ‘erased’ type appears in the error message is when a type error is encountered before the type has been fully determined. I don’t think the compiler elides types in the error message to be more brief, or anything. It could be challenging or impossible to fully determine the type given that it has a type error in it.

[fyl2xp1]

Remy Porter of “The Daily WTF?!” just published a great article Programming is hard. I don’t agree with every single point, but it contains several aspects belonging to this topic.

[diwic]

inferring T: 'x annotations at the type level

I’m not sure this is related, but I’d like to point out that in this code:

impl <'a, B: 'a + Clone, C: Clone + Default + Iterator<Item=&'a B>> Drop for Foo<'a, B, C> {
    fn drop(&mut self) {
        bar(self);
    }
}

…there is a lot of boilerplate and this reduces readability as well as productivity (as you refactor your code and add/remove lifetime and type parameters and have to change these trait bounds in 20 places). This shortened version carries essentially the same information:

impl Drop for Foo { bar(self) }

and is a lot more readable. I’m not sure we can get down to something that small but it’s certainly worth thinking about.

[eddyb]

impl Drop for Foo { bar(self) }

We could probably go down to this:

impl<..> Drop for Foo<..> {
    fn drop(&mut self) {
        bar(self)
    }
}

Removing the .. results in interactions with HKT and it’s perhaps too implicit, but at the same <..> looks really obscure .

[withoutboats]

One of the smallest steps we could take is to elide bounds on the struct on impls that don’t make use of that trait. It doesn’t really seem necessary from a user’s perspective. That is, this seems like it should totally be fine:

struct Foo<T: ToString>(T, u32);

impl<T> Foo<T> {
    fn bar(&self) -> u32 {
        self.1
    }
}

The next step would be to elide bounds that exist on the struct on all impls, so that you could use their methods freely without specifically constraining the type on the impl. This could be a good idea, but the less explicit we get about this stuff the more I worry we’ll lead users astray in how they think about this stuff.

There might be a cleaner modeling of this as a kind of inference of trait constraints from use at the level of the impl. Is there any work in other languages on inferring type class constraints?

[eddyb]

There might be a cleaner modeling of this as a kind of inference of trait constraints from use at the level of the impl. Is there any work in other languages on inferring type class constraints?

Almost anything that’s inference-like tends to pose problems in Rust’s model. Uses of bounds in function bodies must not impact any other part of the crate.

OTOH it’s sort of trivial right now to “inherit” the bounds of the type definition in an impl, although there’s a subtlety where you need to know the Self of the impl which may need the bounds again (but this is being worked on).

In general, the bounds are not different between the struct and impl, if written the same by an user, so just copying them would work, and duplicates shouldn’t be a problem.

[malbarbo]

OTOH it’s sort of trivial right now to “inherit” the bounds of the type definition in an impl, although there’s a subtlety where you need to know the Self of the impl which may need the bounds again (but this is being worked on).

In general, the bounds are not different between the struct and impl, if written the same by an user, so just copying them would work, and duplicates shouldn’t be a problem.

So many places in my code the repeated bounds is longer than the trait implementation. In some modules almost half the code is copy and paste (of the bounds)…

The next step would be

• lifting lifetime/type parameters to modules
  • often you have large blocks of code that share parameters
  • I’ve mentioned a few times I think it would be great to float these to the module level
  • should write up a more complete proposal…

In code that are heavily generic, these two modification would significantly reduces the code repetition and improve the productivity.

[tomaka]

language-level support for AsRef-like pattern

As a formerly C++ developer, this is in my opinion something Rust has totally failed at.

Even though I’ve been using Rust since 2014 I still couldn’t tell you the difference between AsRef, Deref, Borrow, etc. (except for the fact that Deref allows auto-deref).

I don’t know whether it’s a good idea to support AsRef in the language (as I said, I didn’t even grasp what AsRef is for) but I think we definitely need some clarification about these traits.

[arielb1]

Deref is operator*, It has no real semantic meaning, while AsRef and Borrow are more semantic. Borrow is required to preserve hash value and ordering, while AsRef isn’t.

[withoutboats]

Its sort of unfortunate that AsRef/Borrow both exist (with a subtle semantic difference that isn’t expressed in their type signature), but they have a different signature from Deref. Deref dereferences to an associated type, whereas AsRef and Borrow are parameterized, so you can have multiple AsRef impls for a single type.

[iopq]

I got the error messages here:

error[E0277]: the trait bound `impl std::ops::FnOnce<()>: std::ops::FnOnce<(_,)>` is not satisfied
  --> src\lib.rs:43:39
   |
43 |    accumulate(tuples, i).unwrap_or_else(compose(Cow::<str>::from, apply_0(ToString::to_string, &i)))
   |                                         ^^^^^^^ trait `impl std::ops::FnOnce<()>: std::ops::FnOnce<(_,)>` not satisfied
   |
   = note: required by `compose`

error[E0277]: the trait bound `impl std::ops::FnOnce<(_,)>: std::ops::FnOnce<()>` is not satisfied
  --> src\lib.rs:43:24
   |
43 |    accumulate(tuples, i).unwrap_or_else(compose(Cow::<str>::from, apply_0(ToString::to_string, &i)))
   |                          ^^^^^^^^^^^^^^ trait `impl std::ops::FnOnce<(_,)>: std::ops::FnOnce<()>` not satisfied

Maybe experienced Rust devs know exactly how to diagnose the issue, but the error message to me is useless. It just says the trait is not satisfied. Not satisfied by what? Who knows?

fn apply_0<F, V, R>(f: F, v: V) -> impl FnOnce()->R
where
    F: FnOnce(V)->R
{
    || f(v)
}

pub fn compose<A, B, C, F, G>(f: F,  g: G) -> impl FnOnce(A) -> C
    where G: FnOnce(A) -> B,
          F: FnOnce(B) -> C,
{
    move |a: A| { f(g(a)) }
}

This is where you hit the REAL learning curve. Everything seems like it’s FnOnce and it doesn’t compile.

This isn’t even the first time, every time I try to do functions that return other functions, it gets very confusing in Rust.

The problem is that compose has the wrong function type

it should be

fn compose<A, B, F, G>(f: F, g: G) -> impl FnOnce() -> B
    where G: FnOnce() -> A, F: FnOnce(A) -> B {
    
    || f(g())
}

because that function doesn’t take an argument, but the error message is very cryptic to beginners. In my experience, “my program doesn’t work and I don’t know how to fix it” has been a much bigger sink of time for me than “my program doesn’t work and I can throw asterisks and refs at it until it does”.

[rkruppe]

@iopq The error message could certainly be improved, for example by spelling out the : sigil and pretty-printing the Fn* traits

the type impl FnOnce() does not implement the trait FnOnce(_,)

… possibly supported by additional notes and a span pointing at the second argument (where the type that doesn’t satisfy the constraint come from). More generally, many error messages could use improvement. Luckily, there’s already much energy being devoted to that goal (the new error format is now on stable, existing error codes continue to be updated to make use of the new format, and stuff like RFC 1644 is still upcoming). Of course, more is always better!

However, I want to add the caveat that your personal experience with learning Rust is probably somewhat atypical, because you are deliberately and persistently writing in a style (extremely functional and furthermore point-free) that Rust is not designed for and that is very rare in the community. You’re free to do this of course, but that choice does have consequences. For example, I doubt that this specific error will be encountered by many beginners during the critical first weeks — primarily because most won’t even try to write generic higher-order functions.

Naturally, this doesn’t mean the error message is not important or should be permitted to suck, but it shouldn’t be surprising that more effort is put into the most common pain points. While the specific examples you have collected in your experiments with point-free style aren’t necessarily representative, I do believe that there are cases where a common error message is overly cryptic to beginners. RFC 1644 provides lots of extra space for detailed explanations, which will hopefully help, but that’s no excuse not to word the regular error messages for maximum clarity.

[iopq]

There’s a reason this style is rare. When a Haskeller tries out Rust they might try what I’m doing and find it too hard to program in a style they want to program in.

Furthermore, the libraries like tool.rs are great, except for when they don’t work for you because of some higher kinded lifetime issue which you’d never encounter in any other language.

You also need different versions of the same function - FnOnce, FnMut, Fn and probably have to adjust for arity as well.

Overall, the support for functional programming is barely passable. I don’t see why that has to be the case. Rust SHOULD support functional programming for the cases where it’s desirable.