Why is it a problem that should be solved by tools?
Sure, it’s a problem that can be solved with tools, like a wide variety of language usability issues. As you say, Java can be quite productive despite its verbosity. But a lot of people also hate Java for that verbosity (including me), because, among other things, (a) code is read more than it’s written, and it’s harder to determine what a piece of code does or whether it’s correct if it’s full of useless fluff, and (b) not being able to sanely write code without an IDE is annoying.
You could just as well say that (like Java) Rust should not have type inference because you can just have the IDE complete all your types.
I do not mean to imply that I don’t think tooling is ever the answer. When explicitness makes code easier to read rather than harder, I agree tooling can be a good approach to help programmers deal with the resulting cognitive burden (as long as it’s not effectively mandatory). Indeed, rustc on the command line already provides suggestions, although it is not so nicely integrated as an IDE. But it’s hard to argue that a spew of derived trait names passes that test.
Agree, my statement of "the problem of productivity and learning curve should be solved by tools" is wrong, because this problem actually consists of a ton of subproblems, and only a part of those problems could be solved by tools.
But I stand by the
adding alternative ways to do something just because the proper tooling is nonexistent at the moment is a bad idea.
statement. I find it disturbing that for a lot of small cases a language solution is proposed, and the tooling solution is not even considered. This is understandable, because changing a language is easy, there is a compiler which you can just hack on, while providing the tooling is very difficult, because the basic infrastructure for inspecting and editing rust code is nonexistent.
As an example, let's consider this derive problem again. Here is my simple solution: just add an action, which is active when you are inside a struct definition with "value" fields, which adds all necessary "value" derives. Can you write a hundred lines of code to implement it? Looks like the answer is no, because where should you add those lines? To the compiler? But it knows nothing about editing the code, and it is not interactive. To your favorite editor? Well, there are 2 + many editors out there, and none of them is capable of doing this properly, because they know nothing about syntax and semantics of Rust.
You can't implement a simple (hundred of lines if you have the necessary infrastructure) tooling solution, so you necessary need to push for a language solution. And this is a problem which is worth solving imo.
I agree that tooling is not always the solution (though I think that at least for 4 problems I have described in the post the tooling is the best solution), but the current problem with Rust is that it is never a solution.
EDIT:
And, for example, to solve derive problem in Dart, you would need to write those 100 lines here, and it would be available in any editor.
I'm writing this on my phone, so I'm sorry for any and all overlooked spelling and grammar mistakes.
On the topic of solving by using "tools", I'll agree wholeheartedly if we consider macros tools. As for explicitly relying on a set of external IDEs, I'm not sure I think that's the right solution for many of these issues.
This! As someone who spent many of my early months in programming Racket I can't even begin to express how well the family of language concept works. This is why I was pushing on the Macro 2.0 to avoid ! in the names, since it would allow macros to become a tool for sub-language development. I personally find myself wanting Rust-like types without Rust-like ownership and static allocation. Rustscript (for lack of a better name) would be a killer feature. This isn't even really a new concept in Rust, as we already have unsafe.
fn foo() {
script {
// Could `new` be a macro?
let x = new Foo;
let y = x;
println!("{:?} {:?}", x, y);
}
}
In response to the OP though:
I personally never find this all that hard to deal with, but I don't spend a lot of time interoperating with C libraries in Rust at the moment. I like that my base numeric types feel very un-magical.
Figuring out what language support is needed from Rust to have a nice object system (might be using the wrong terms here) is key. Every project I've written with a complex data model has eventually needed something like this in places. Bridging the gap between the relational mental model and the functional immutable model could be really interesting in Rust. My only real fear is that doing this in a way that upholds the zero-cost or very clear cost is going to be hard.
Yes please.
Interesting. I actually really like this idea, the general type parameters often give you a high level sense of the kinds of places you might want to use something, having this show up once at the module level seems like a readability win in a big way.
Kinda related but, I always find myself needing to explain the orphan rule carefully, and the reasoning isn't completely clear at first to new users.
I think that concepts from Design by Contract should be considered as a part of this again. Racket for example has a really good story for runtime error messages. I would be sad if Rust aims for anything less.
Two kinds of programming language I want; Magical, and Non-magical. Rust is very clearly in the non-magical category, or should be.
I also want to add that despite that the module system is one of my favorite things about Rust, it can be a bit of a pain point in the learning process. I'm not about to propose something as crazy as automatic name resolution or something along those lines, but I can definitely see the concept of re-exporting types for the purposes of modularity and privacy being new to people.
My point was that in all cases, I want to see an explicit list of the traits being derived, in the #[derive()] macro. I.e. I don’t want to see an aggregate or shorthand, because I find it less readable. The focus here being on what text I want to see in the source file, rather than how that text was generated. Some verbosity is bad and some is good.
I disagree about macros, because macros in a lot of cases are crutches for missing language features. println! is a good macro, because it does something that’s non-trivial. A lot of macro usage in Rust is like “Rust doesn’t support type level integers, so just macro it up to the 10 case and wait for the new feature” or “it’s awkward or impossible to write this kind of polymorphic code in Rust, so just write a macro and pass types into it”
In other words, if your macro is doing the work of a C++ template, that means the current generics implementation is not working for you.
for (i, j, k) in iproduct!(0..4, 0..4, 0..4) {
for (i, a, b) in izip!(0..100, &mut xs, &ys) {
You can't add everything to a language, so having a meta-feature that allows you to build very specialized features is probably handy.
On the other hand, variadic typesafe generic functions as implemented in D language are useful in sufficiently common situations.
There are reasonable persons that think that C++ templates are used for too many things. You can argue that more specialized and more typesafe generics are better than the "dynamically typed" (at compile time) templates of C++.
That’s what I’m saying, macros are not as safe as generics. There should be as little as possible macros and as much as possible generic code. This is because generic code can be type-checked at declaration and give nice error messages.
Right, but keep in mind that if you want to implement generics (and variadics) as powerful as in D language while keeping type safety (D generics are not type safe at compile-time), you have to introduce several higher level type system features that today you find only in Haskell and few dependently typed languages (I am not sure even Scala is powerful enough). Perhaps Rust will gain such features someday, but it's a big complexity budget. In the meantime Rust macros allow you to do something similar in a less type safe way with quite smaller type system and language complexity (and usage difficulty, with much less "type-astronautics"). As it often happens it's an engineering trade-off.
I think that is an example of people using macros to get around a problem of the laungauge at the moment. That seems like a really really good thing to me. Can you imagine how much more irritating things would be without macros.
Macros aren’t ‘less safe,’ in that they generate code that has to be well typed, but because type checking is delayed to expansion, its very possible to write a macro which will generate invalid code under some circumstance which you didn’t test for. So if your library is based on macros, it can easily be invalid. This isn’t true of code based on Rust’s generics system (but it is true of D & C++'s template systems)
Obviously coming from a dynamic language like Racket, this may not seem like a big problem.
Ok I see what is meant here by safe (though I’m not sure it’s the right word). Exposing macros as a library author should be viewed as very different than exposing other forms of abstraction for exactly this reason. Between this and the fact that macros don’t always perform expected control flow, and that macros can lead to strange type errors at compile time. This has been my rational for the ! in the language since I asked about this on the macro RFC.
With all that said however, macros are still syntax extensions which let me change the way I write Rust more fundamentally than all of the other forms of abstraction. I’d like to believe that a language like Rustscript could be implemented on top of Rust as various macros and functions, while still resulting in an ergonomic new language.
Yeah, the ‘being generic over numbers’ argument is the strongest IMHO. Another one: Macros get expanded up front, whereas generics survive up until code gen and thus can receive MIR optimizations once instead of once per call site.
Since we want to allow someone to derive Eq but manually implement PartialEq, rustc can’t just always generate an impl for it. And rustc doesn’t have the type tables yet to check if it exists (derive runs before then).
Those issues are all implementation-related. I think the complexity of language implementation doesn't really matter. It's a problem for compiler developers. We threw away the idea of having a simple compiler a long time ago when we went with LLVM.
Why do these two things have to be in conflict? I'm disappointed to see this assumption unchallenged.
"Be explicit about everything ever" was never a goal of Rust. We often tried to err on the side of explicitness when we weren't sure what to do, or when we knew that overly implicit behavior caused real problems in other languages (e.g. super-liberal numeric conversion in C). But there have been plenty of instances where we added inference and it made a huge difference in the expressivity of the language. A couple of examples that most people here likely don't remember:
Moves based on type. You used to have to write move before values when moving them. This was tremendously inconvenient. The idea of implicitly moving was incredibly controversial; a lot of people thought it wouldn't work at all. But nowadays nobody questions it.
Capture clauses. You used to have to explicitly choose whether to capture all variables by value, by immutable reference, or by mutable reference. This added a lot of verbosity to closures. People thought that inferring captures was out of the question, but it actually turned out to really enhance the usability of the language.
Nullary-enum-variant/variable disambiguation. It may be hard to believe now, but to deal with the ambiguity between match foo { None => ... } and match foo { x => ... }, you had to write . after enum variants: so for example you would write match foo { None. => ... }. If you didn't do this, the compiler would think None was a new variable binding and you'd get a confusing error. When I proposed just inferring, people swore up and down that this would ruin the language, that the compiler complexity would mushroom, that it would be incredibly confusing, etc. etc. None of this happened, and nowadays the controversy is entirely forgotten.
The lesson of history, to me, is that people worry a lot about implicit behaviors, but as long as thought is put into the design and lessons from the implementation are taken into account bad interactions rarely happen.
I have been reading and thinking a lot about the topics on this thread. I think that this comment by @WiSaGaN is very interesting. Even though I think that @WiSaGaN was basically pushing back against many of the proposals that I support, I actually agree with a lot of what they are saying here:
In particular, I agree with the end. We do not want to make "ergonomic improvements" that make it harder for programmers to spot easily noticeable bugs or to control performance. In other words, while I want to focus on productivity, I do not want to do so at the expense of reliability or performance. What we should strive for are positive sum changes that improve productivity without costing us elsewhere.
I think where @WiSaGaN and I may differ is in two areas:
How feasible is it to find such positive sum changes? I feel like when I look back over the history of Rust, examples pop up all over the place. We've seen a number in this thread, but I'll dig into one particular example in a bit.
Who will benefit? It's easy to think that new users want to ignore details whereas experienced users want to control them. But I think that's wrong. I think that what all of us want, new and experienced alike, is to control the details when it is important and ignore them the rest of the time. So we should strive for ways to give people control, but in as simple and minimal a way as possible.
A case study: deref coercions
There have been a number of examples thrown out, but I'll add another one: at some point, we added "deref coercions", which basically allow you to write &x instead of &*x or &**x and so forth. The nice thing about deref coercions is that they allow you to think about your code at the level of ownership and borrowing -- when you call a function, if you want to lend that fn access to your data, you can write foo(&x). If it happens that this function wants a &str and you have a Rc<String>, that's no problem. Before we added deref coercions, you would have had to write foo(&**x). Clearly this is more explicit, but is it adding any value?
Certainly I am someone who feels pretty comfortable with the concept of pointers. But one of the things I hated most in the pre-deref-coercion days was having to add just the right number of * annotations to make things compile. It wasn't like it was hard for me to do, but it interrupted my flow. I had to stop and look closely at the error message, get the types just right, and never once did it make me say "oh! I thought it was just foo(&*x). Now that I know it is foo(&**x) I will try to do something else". So for me, as an experienced Rust user, this change was a big usability win.
It should go without saying that being extremely particular about &x vs &*x vs &**x was also not helpful for new users. There the problem is compounded. Not only is the compiler being fiddly with little value, but the new users don't have all the concepts firm in their mind. They are still trying to understand ownership/borrowing but now they must simultaneously understand a few other things. If they encounter enough such errors, there is a good chance that they just walk away. So for new users, this change was a big win too.
This is not to say the change has no flaws. For example, I have occasionally -- well, I think exactly once -- read some blog post that complained that Rust doesn't make all your pointer dereferences explicit (this also, of course, applies to ., which btw does all kinds of things to make method calls ergonomic). While true, I don't think that writing * explicitly really helps to control performance except perhaps in some very exceptional situations -- and even then the compiler's optimizations probably clean it up for you. Mostly, autoderef and deref coercions just make things more DRY. You as the programmer specify your data layout in struct and enum and fn declarations. The deref coercions fill in the gaps, for the most part. To me, this is an example of controlling the details only when it's important (in the actual layout of data) and not so much when its not (in your local variables).
Another problem is that coercions always have a bit of janky interaction with generic functions. If I have fn foo<T>(x: T) and I pass foo(&x) where x: Rc<String>, T will get inferred to &Rc<String>. Sometimes you would have preferred that T be inferred to &str, perhaps because of trait bounds. This is a tricky problem. I think we can do better than we do now. But on balance, I think deref coercions totally paid off -- and maybe we'll make progress on improving the edges.
Generalizing from that example
Many of the examples that I added in my original list fit this mold. For example, match expressions do not autoderef, and require explicit ref keywords to create references. This means that if I have, say, an &Option<String> and I want to get a reference to the string inside, I have to write some code like this:
match *x {
Some(ref v) => ...
}
Interestingly, the * and the ref here are basically mandatory. There really isn't anything else that would even type-check! At this point I usually write them first, and if I get a type-check error, of cousre I know how to fix it, but it's never the sort of error that I really appreciate, that reveals a flaw in my thinking. It's just busy work. It would be much nicer if I could write:
match x {
Some(v) => ...
}
and then just have v (which would have type String) be an implicit reference into the content of the option. This would retool how we think of match statements quite a bit. The mental model would shift from "a binding v moves the value out of the value being matched" to "a binding v creates a reference into the value being matched, moving only when v itself is moved" (more or less). @nrc and I have worked out a lot of the details here and I think it works out quite nicely (there is one catch about cases where you assign to x within the match arm, where we would also have to force moves, though I'd probably lint). It's basically the same as closure upvar inference -- a closure borrows things from its environment if it can, moves if it must.
Presuming @nrc and I are right, and this can be done, then I posit that making this change would make the experience of using Rust smoother for experienced users. For new users, I think the effect is more dramatic. With today's language, one has to simultaneously learn how match and enums work (which is unfamiliar for many people) while also balancing that with ownership/borrowing. This change would help to separate those concerns, so they can be learned independently.
Conclusion
This is turning into more of an essay than a comment, sorry, so I'm going to stop here. The main themes I wanted to emphasize:
We should not sacrifice control, but identify way it is truly needed and where it is accidental.
In terms of improving new user experience, we should look for places where multiple difficult concepts coincide.
In terms of improving end user experience, we should look for places where you have to write things that have only one reasonable answer, and see if we can avoid that (sometimes we won't be able to, sometimes we will).
Certainly updating the language is not the only way to improve new user experience. Tools can help. Documentation can help. But the language is a tool to be brought to bear too. =)
In general I am excited to read the details of these proposals, and I am optimistic about them. But I do want to add one caveat. The comparison to existing ergonomics features are compelling, but its worth mentioning that these features have not been pure win. I think they can create a situation in which less informed users are unable to model the language semantics in their head, because they can’t figure out why some statements are valid and some aren’t.
Move or copy by type, for example, can create a situation in which a user who doesn’t understand Copy doesn’t understand ownership. Sometimes their values are moved, but sometimes they aren’t. What gives? I’m glad Rust has this feature, but it does have a cost for learning.
For deref coercion, I remember my own confusion very concretely. It wasn’t so much deref coercion itself, but that there are all kinds of convenience impls in the standard library like all these by reference impls of Add or how many traits are implemented for references to types that implement that trait. As a new user who had learned about deref coercion, it seemed like deref coercion was happening all over the place, when in fact it was just the standard library. So when there wasn’t a convenience impl, my mental model was shattered and I was confused and sort of helpless. I still get bitten by gaps between my mental model and Rust’s real semantics every once in a while.
I think its important, but challenging, to think about it from this perspective. How will a user without a pre-existing understanding of the language build a model of its semantics from their experience using it? How do we shape their experience to guide them toward a model that is consistent, useful, and correct? I think these proposals are all very plausible, but I wish we could get better data on this perspective. Nightly doesn’t really cut it here.
Definitely. And I did talk about that in my comment as well: in particular, I mentioned the interaction between deref coercions and generic types, which can be confusing, which I think falls into this category. Deref coercions have the side-effect that sometimes you have a fuzzy model and eventually that must be made precise.
Naturally I think we should try to minimize these dangers. But I also think a piece of the puzzle in addressing them may be documentation and more advanced tutorials and the like. I think right now we have a lot of documentation, tutorials, and the like aimed squarely at beginners, and much less bigging into these more advanced topics.
Ideally, we can get people far enough along in Rust that when they start to hit those corners, they are ready to deepen their understanding and learn how things work. If all we have managed to do is push the sharp corner from the first 5 minutes to the first 15, then it is probably not a good tradeoff. =(
(With respect to the convenience impls for &T, I also sometimes get confused about "where do I need a * and where can I avoid it" and so forth. I am not sure of the best remedy here. It may be that we just need a few more impls to make this come up much less often. Or, it may be that we could add some auto-deref at various points to achieve a similar goal. Or maybe there's not much we can do. I'm not sure. (I actually listed precisely this confusion in my initial list... e.g., map(|x| x + 1).filter(|&x| x > 2)))
