[Roadmap 2017] Productivity: learning curve and expressiveness

[rkjnsn]

Closures in argument position get better inference.

This in particular always struck me as odd and unnecessary. It seems like the complier should be able to infer the type of the variable from its eventual use as an argument, and flow that back to the closure definition.

[scharris]

I guess it’s a personal preference, but I like that sort of transformation chain that iopq provided as an example, more so than if the intermediate results had been named.

When the closures are short and obvious like that, and the closure arguments are given proper names as in the example, I really don’t think additional names for intermediates are adding anything for the reader.

To me the chaining also helps to see at a glance that each stage only depends on the output of the previous, and that no intermediate results are being used anywhere else in the remainder. If you have named intermediates and you want to make changes to various stages, you have to first make sure the existing intermediates aren’t being used in the rest of the function.

[burakumin]

Concerning expressiveness of Rust, I had proposed an improvement for implementations and composition. Most of the people who commented expressed their enthusiasm for the idea but I’ve yet to receive feedback from the language team.

Rfc: delegation of implementation

[withoutboats]

I’ve personally been thinking a lot about this RFC lately. I think some delegation mechanism is needed, but its sort of tricky. In addition to the ‘composition delegation’ concepts you cover in the RFC, there are also desires to enable delegation between trait impls and inherent impls. I’ve been meaning to leave a comment on the RFC thread, but I’m trying to figure out how to balance these different needs.

[burakumin]

I’ve personally been thinking a lot about this RFC lately. I think some delegation mechanism is needed, but its sort of tricky. In addition to the ‘composition delegation’ concepts you cover in the RFC, there are also desires to enable delegation between trait impls and inherent impls. I’ve been meaning to leave a comment on the RFC thread, but I’m trying to figure out how to balance these different needs.

I have recently added possible extensions although they may not cover all the cases you’re thinking about. Now I’d be very interested in hearing which scope you would like to cover, what are the tricky cases you have in mind and what is the impact on the possible manners to handle delegation.

[eddyb]

The reasoning is tricky but it is a necessary hack if you have ordered type-checking: that is, while rustc employs HM type inference, it also has type-dependent resolution (things like fields and methods) and it traverses the function in AST order, so when it needs information, it can only know the results of HM inference for the parts of the function that come before in the source.

To solve this one must move to an order-independent constraint-based type-checking system: just like we have trait bounds checked in whatever order they happen to become known (i.e. enough inference information has been collected), we could have every single type-dependent construct kept around until it can be resolved.

This would be a significant rewrite, but strictly an improvement, if Rust didn’t have coercions.

The problem with coercions is they don’t primarily rely on HM inference (if you infer too deeply then you end up with a trait bound failing instead of a later coercion), although they do use it to understand e.g. function calls, their system is instead an “expected type hint” passed down from parent to child and transformed as needed.

This is the only kind of “eager top-down type propagation” rustc can do - the expected type reaches the leaves first, and only then HM can propagate what it might have learned from what was expected.

This hybrid system is problematic because of relatively determinism: how do you keep the same kinds of code working, i.e. how do you do out of order coercions withiut accidentally leaking some type further than they should reach (e.g. preventing a coercion in the only place it could be applied)?

Swift’s answer seems to be bactracking, and you can find discussions around (sadly I don’t have one on hand atm) about implicit type conversions resulting in potentially days of compilation time (backtracking being roughly of factorial time complexity while only linear space usage, so you can’t easily run out of memory, as I’ve had the recent unfortune of finding).

The most promising solution IMO is a boring fixed-point algorithm that repeatedly tries to resolve constraints it can be certain of, and when it runs out of those, just picks the first (in AST order) coercion to resolve to identity, which will hopefully unlock other constraints, without doing so in a way that would be backwards-incompatible.

This might be something we pursue in 2017 but not at the cost of compiler performance.

[thepowersgang]

Re the fixed-point algorithm. That sounds quite a bit like how I’ve structured type checking and inference in mrustc

It works rather well, but has required a few patches here and there to properly cover all of the code rustc can check. From my rough testing with some cases that people on #rust have complained about, it manages to correctly infer some cases that rustc fails on. A disclaimer - This has only been tested by compiling libstd (and all dependencies), so there may be places where it falls flat on its face.

A rough layout of how it works:

• An initial pass over the function’s AST to both give IDs to all unknown types (_) and build up a set of known equalities (applied using HM type inference), coercion equalities (stored for later checking), and trait bounds (including operator overloads)
• Iterate until there’s no changes in the ivar set:
• Run code to handle method lookup, indexing, value calls, … (anything that needs auto-deref)
• Check trait bound rules (which can provide the type of an associated type)
• Check coercion points (recording the possibilities for ivars, applying type equality if a coercion is impossible, or inserting the coercion operation if required)
• If nothing changed above, Use type possibilities from coercion rules to propagate type information through coercion points.
• If nothing changed above, replace literal types with the defaults (i32 or f64)
• Check that there are no rules left, and that there are no unknown ivars left.

(For the brave, see src/hir_typeck/expr_cs.cpp)

[glaebhoerl]

(@thepowersgang the link has a typo)

[thepowersgang]

waves hand no it doesn’t (Fixed, thanks)

[sanmai-NL]

I agree, it would help recognizibility if the most important part of the signatures were highlighted, for example using a bold font, instead of the least important parts being hidden.

[leonardo]

Language usability has recently improved a bit, becoming a little more DRY. This is the old version of Rust code:

struct MyStruct { data: u32 }
impl MyStruct {
    fn new(data: u32) -> MyStruct {
        MyStruct { data: data }
    }
}
fn main() {}

With the latest Nightly you can write it like this:

#![feature(field_init_shorthand)]

struct MyStruct { data: u32 }
impl MyStruct {
    fn new(data: u32) -> Self {
        Self { data }
    }
}
fn main() {}

[llogiq]

Could we auto-derive a constructor method for structs that just takes all contents as arguments – basically #[derive(new)]?

[leonardo]

Could we auto-derive a constructor method for structs that just takes all contents as arguments – basically #[derive(new)]?

You should write an enhancement request for this nice idea

[tomaka]

Structs where #[derive(new)] would apply should have all their fields public in the first place.

Usually you write constructors, getters and setters instead of using public fields in order to be able to change the internal representation of a struct without modifying its API. That’s not possible if you use #[derive(new)] as you’d break that API.

Plus if you can write a dummy constructor for your struct, that means that there’s no possible invalid state for the values of the fields of the struct (if there was one, you’d need to check in the constructor, which you can’t with derive(new)). Therefore there’s no need to have private fields for that struct.

[gnzlbg]

#[derive(PartialEq, Eq, PartialOrd, Ord, Copy, Clone, etc)] there are a lot of fine-grained traits to derive easy to forget something; annoying to have to have both PartialEq and Eq

maybe shorthands for common combinations? note: custom derive may allow experimentation here

C++'s Concepts TS has Concepts like Regular, SemiRegular, etc., where SemiRegular is a type that behaves like a primitive type (e.g. i32) in terms of copying/moving and Regular is a SemiRegular type that also behaves like i32 in terms of comparisons (eq/ord). So I can imagine that we could in Rust have something like:

• SemiRegular: Copy + Clone,
• PartRegular: SemiRegular + PartialEq + PartialOrd, and
• Regular: PartRegular + Eq + Ord,

or something similar to this.

• lifting lifetime/type parameters to modules

• often you have large blocks of code that share parameters
• I’ve mentioned a few times I think it would be great to float these to the module level

Are you talking about ML parametrized modules? That would certainly help ergonomics but it is also a big new language feature.

Some new features we might consider

This is the perfect list of features Rust still needs, agree with every single one of them. The only ones missing are variadics and, well you mention RFC 1598 but ATC/HKTs, or something in this direction that solves most problems would be nice.

[nrc]

Could we auto-derive a constructor method for structs that just takes all contents as arguments – basically #[derive(new)]?

Yes - https://crates.io/crates/derive-new

[llogiq]

Not neccessarily – a derived new(..) would be indistinguishable from a manually written one, at least from the outside. So we might require Default impls for all private fields and only use public fields as arguments for our constructor.

Later if we want to require some processing or validation, we can replace the derived constructor with a custom one.

[cessen]

• lifting lifetime/type parameters to modules

  • often you have large blocks of code that share parameters
  • I’ve mentioned a few times I think it would be great to float these to the module level
  • should write up a more complete proposal…

• inferring T: 'x annotations at the type level

  • these are just kind of annoying and nobody has a good intution for what they mean
  • I think we could infer them, just want to experiment to be sure it doesn’t lead to confusing errors later on
  • on the other hand, the compiler tells you what to type, and you type it, maybe not so bad?

I feel like both of these are part of a more over-arching issue: lifetimes can often be annoying to work with.

From a (shallow) ergonomics standpoint, it would be awesome if Rust could just infer all lifetime parameters, and only shout errors at you when the situation is unresolvable for some reason. However, there is at least one significant problem with doing that: you really want lifetimes to be explicit at API boundaries, because they are actually part of the API.

My understanding is that this is the primary rationale behind requiring explicit lifetime annotations for e.g. function signatures, data structure declarations, etc. All of those things amount to API boundaries, and thus lifetimes are something that should both be thought about and made explicit.

But are functions etc. really Rust’s API boundaries? It seems to me that the real API boundaries in Rust are modules. Specifically, public members of modules. It would be amazing if Rust could infer lifetimes in private function signatures, private data structures, private traits, etc. This would make it significantly easier to write e.g. helper functions for code with complex lifetime requirements.

In fact, arguably, maybe the real API boundaries of Rust are crates. Maybe lifetimes could be inferred for all private things within a crate. Then people would only (strictly) need to write lifetime annotations for the publicly exposed parts of crates. This could significantly simplify e.g. writing application code, among other things, while still enforcing that crates are explicit about the lifetimes in their API’s and therefore don’t accidentally break their API promises.

So, having said all of that… I’m not actually advocating for anything in particular here. There are probably issues or complexities with what I’ve written above that I’m not aware of. But I wanted to put these ideas out there to maybe spark some discussion of how Rust can make working with lifetimes less of a hassle in a broader sense, rather than just as a set of one-off use cases.

[kornel]

My understanding is that this is the primary rationale behind requiring explicit lifetime annotations for e.g. function signatures, data structure declarations, etc. All of those things amount to API boundaries, and thus lifetimes are something that should both be thought about and made explicit.

BTW, this design principle also gets in my way when I don’t have any API boundaries that I care about when I write binaries. I totally understand API boundary issues for libraries, but the necessary constraint for libraries is a burden when writing a program where everything is under my control, final and de-facto private.

In my programs I often want to use traits simply to reduce repetition, but Rust’s explicit API rule requires me to declare everything up front about type parameters (instead of at point of use like in C++), so working with generic types causes explosion of where annotations. Even to add two things together I have to declare use of Add and probably Output of Add too (and I keep confusing syntax of T:Add::Output and Add<Output=T>, etc.)

I also instantly regret whenever I add new fields to structs. If the struct didn’t have a reference before, it breaks the whole program. I once added a HashMap to a struct, and had to add : Copy + Eq + Hash + 'static in 14 places in a 150-line file

So I’d love if Rust inferred everything that is private/module-scoped, and did what I mean, not what I explicitly declare.

[est31]

Regardless of where you think the real API boundaries are, the more code you have to scan to find out a type, the worse. Code is read more often than it is written. And I’m seeing already now when this feature gets introduced, not specifying the type becomes “idiomatic” rust, clippy starts suggesting to remove types because “its unneccessary” and suddenly most of rust code becomes unmaintainable because you don’t know anymore which types you use and which you don’t.

I want to know which type something has, and fixing the types at the function border is a good compromise IMO.

Maybe some people would like a dialect of rust where all traits, lifetimes and types are elided at compile time even at the API border, where you don’t place braces nor semicolons, but its not something I’d consider useful for my purposes.

Possibly putting the type into the function border can be some task for an IDE, if people really think that not thinking about types makes you more productive (it doesn’t because the less you think about stuff early on, the more you’ll regret it later on), but I don’t want to have to use an IDE to find out which type something has.