One thing I haven't heard mention yet (sorry if I missed it) is visibility of existing resources. When I mentioned learnability on Reddit people suggested e.g. tourofrust.com, Rust in Action and cheats.rs. But how do new users discover these?
Creating more learning resources is a great goal (btw, and more videos would be good too) but if they aren't visible to new users then we'll still have a problem.
Another topic to think about: learning cliffs.
In working on Rust, I've seen several instances where an experience forces someone to confront a large set of unfamiliar concepts, and they give up. For example:
A person wants to do an ownership-sound action, like taking disjoint mutable slices of an array, or temporarily taking ownership of an &mut value. This is simply impossible without knowing the relevant helpers (Vec::split_at_mut, take_mut), or without implementing it yourself in unsafe code.
A person wants to understand why their warp code isn't compiling. They look at the type definition of the combinators, and have no idea what an HList is or how any of this works. Without understanding type-level programming in Rust, they have no idea how to interpret these traits.
Learning cliffs often arise when the details of a system's implementation are needed to understand its interface. For example when the simple ownership rules diverge from the actual borrow checker. Or when a simple mental model of a primitive's type signature diverges from its implementation. Compile-time cliffs are particularly frustrating, because you have code that simply won't compile. You can't look at the output, debug it, or otherwise interact with the compiler.
Helping users understand when they've reached a learning cliff and what to do could be valuable. Eg directing users to better high-level explanations, better in-depth learning materials on system implementation, or better APIs that avoid the issue.
As the other co-author of The Book, I also wanted to add some ideas, thoughts, and feelings.
Overall, I love this idea and think this is important work!
As far as this interacts with The Book, I feel... less excited 
I think Steve and I have a really good idea of the rough parts of the book. Fixing them, though, has a few problems: for one, neither of us have much time to give to the book these days. Secondly even if we did have time, the book feels kind of like spaghetti legacy code at this point: whenever we try to fix a problem in one spot, we inevitably cause problems in other spots because of how the topics intertwine, the examples build on each other, and so forth. Steve even started experimenting with a new book at one point (sorry @steveklabnik , I can't find a link to it right now). Prepping large edits for being incorporated into the print version is also a LOT of work, and not incorporating them causes confusion about why the online and print versions are different.
All this is to say, please don't generate more feedback for Steve and I, please. In fact, the issues and PRs on the book might be great data for you, but we really don't need any more. And if the book is no longer meeting learners' needs, or not meeting all learners' needs, please consider creating new resources, not modifying the existing one. I often tell folks how glad I am that the O'Reilly Programming Rust book, and @timClicks' book, and other books, exist, because they're all different and that means The Book doesn't have to try to be those books too.
Jake and I also made the Rust in Motion video series to address the problem we kept seeing of people who are used to being able to pick up new programming languages by just diving in, and they hit a huge brick wall with Rust. We focused it on the parts of Rust that are most different from other languages (ownership/borrowing, error handling, lifetimes) so that you have enough knowledge of the concepts to get back to however you usually learn languages.
@shepmaster has also done a number of trainings and is also prolific on stack overflow, so he might have some useful thoughts as well.
I completely agree with this: it will be literally impossible to have the one book that rules guides them all; each book should have a somewhat opinionated stance and teaching style, so that it will best fit some people, whereas other people will prefer different learning materials.
For instance, regarding my previous post, I do think that it is important to have a learning resource that is as beginner friendly as possible! That's almost the most needed learning resource in any topic. But now that we have it, I do think it is time we enhance the official learning resources out there with more precise (and thus maybe more intimidating) ones.
I feel I struggle to learn and comprehend Rust because of lack language understanding. Tecnical english can be hard to read at times when you dont have it as main language or are not educated well enough. To aid this I think animations, illustrations and graphical figures to represent and visualize concrete and abstract consepts would be a tremendous help. A lot of people learn better by visual input rather than reading to comprehend stuff. Visuals are universal, written language is not. Visuals should definetely be a part of the Rust book.
At the same time I also miss a simple way to make graphical interfaces. Some cool stuff is on with Bevy for example. I really think more people should involve them self in that with the aim to make a cross platform lib built into the language.
A lot of people also find it more rewarding to see a graphical result of their work if it is easy to do it.
I had a great time learning about programming using Flowstone, but it is purely 32bit Windows and not really a programming language. It is graphical/visual programming with the ability to use premade blocks/items/functions/whatever and the ability to make own code in a C like language, ASM and Ruby. But you can make pretty much anything with it. It is in my opinion very easy to learn. And the results comes very quickly. I would love to se something like Flowstone based on Rust, to develop in Rust. Have a look at the Flowstone manual. Or the component refence.
.
But for a start please make things more visual orientated in all aspects, especially in the Rust book for learing this cool language. <3
This is a great point that I'd like to make clearer: We only aim to provide feedback where it is wanted and needed. I apologize for using the book as example (even after Steve gave similar albeit a little less on the nose feedback). I certainly have 0 interest in causing more work with no gain for anyone. 
I also have no interest in prescribing fixes to specific learning materials but rather to provide evidence about how groups of people learn Rust in various settings. We may actively ask learning material creators if they want to work with us on gathering this evidence, but we won't spam folks with specific issues about the evidence we gather.
My plan for the start of the new year with regards to this effort:
If you're forming a working group for this, I'm interested in joining I'm not sure the process for doing that, though.
First of all, I appreciate the desire to open such a group.
I have some feedback to this, some pretty fundamental. My core criticisms:
While I appreciate the desire for a user study, the subject matter, especially around programming languages is what I call a research problem[1], which does not seem to be a good first task for a group. It is very hard to assess and de-bias such a dataset. Self-Reporting is also highly problematic for detailed conclusions.
I think making the language "better learnable" is a good desire, but there's lower hanging fruit. My favorite example here is the turbofish-Operator: while it is a wart in the language and might trip people, it has no interdependence with other parts of the language and can just be learned as an odd fact. Science Educators teach such facts easily every day. The other side is that Rust has a number of fundamental concepts that are very new to people - for example: is Ownership or Borrowing the dominant concept in Rust? How do we get people to model with Ownership quick?
Honestly speaking, on almost every RFC where "learnability" ended up the final argument, my assessment as an educator was that the change was not worth the effort and cost of backwards compatibility.
We do, however, have a pool of knowledge already within the Rust project even: by speaking directly to educators and collecting practices. We do have people that taught Rust over the last 5 years, in settings from small to large groups. The have already tested many approaches. I think it's a way faster path to success for such a group to start speaking directly to or build it out of those people and have them condense their experiences into supplement material. Rather than changing "the book", producing co-material for those interested in teaching would be a huge boost to the ecosystem. Also, it can provide a number of sources to those with the desire to create courses.
This has multiple advantages:
a) It draws educators into our project and gives them a place. This forum can also be used for feedback.
b) It makes the group operate on what is currently available rather then having the group set up a foundation first.
c) It is common practice in many organisations that revolve around training/teaching, e.g. every sports association as a trainers forum rather than a "what do we change about the sport to make it easier?".
Currently, the effort of producing teaching Rust means reinventing the world every time - addressing this may yield much better results much faster.
[1]: In that academia is still running studies for assessing those topics.
We have been running a survey for Rust users internally on Fuchsia with 70+ respondents, with a focus on identifying pain points and how people learn. If I get permission I'll make sure to share it.
A couple of observations, that may be counterintuitive:
Also interested in joining!
Here is how I love learning - by playing first!
Show me the end product first. Let me copy paste the commands to build it on my laptop. show me how to make a small code change and re-build it. This will make me feel great within 5 minutes and it will also show me how a real world codebase of Rust looks like.
Than dive into each parts of the codebase and while you explain them, tell me about the feature of the language (struct, functions, borrowing, etc).
Almost all the education material I see takes the bottom-up approach and I fail to understand why. It's the same problem in CS courses and in free resources online. Look at the Table of Contents of most books for learning Rust - they have the language features. Instead they should have the features of the app I am building as the first class citizen. Sure, the book/tutorial should also have the features of the language (or links to deeper docs) but the focus should be on PLAY/build/have fun.
I like the initiative but not the direction.
I do not believe that asking people what are their problem with learning can be used as an guidance. It is like asking what the phone should look like in pre-iphone era. Surveys will direct you wrong way. At the same time we do have users forum which contains endless amount information on what people struggle with and why they ended up being confused or were not able to find relevant information. Also, there were some threads discussing this exactly question, if you have mastered Rust, how did it happen? Those are way more valuable because it contains reflection of people who are already knowledgeable in Rust and thus it makes it much more valuable source of insights. Just search "learning rust" on the forum. Is yet another survey expected to bring more objective result than this? Search results for 'learn rust' - The Rust Programming Language Forum
Goals sound a little bit strange to me. First, "information obtained by the project group to be useful ... as well as to Rust educators". I do not see how this is possible. Shouldn't the help be directed the other way? Educators are the ones who have first-hand knowledge about learning problems and they gain this knowledge directly from the students, whereas this project is going to gain it from surveys. Surveys are much more inferior source of knowledge than personal experience.
I do not understand what this one is at all. The first precondition for any project to succeed is to have concrete goals and success criteria and I do not feel they were set here. How do you evaluate either "assisting" was successful or not?
Questions to be researched are either already known or irrelevant. For example, why does it matter which background makes it easier to learn rust? Are we going to tell Javascript developers, sorry, your background will make it harder, we wish you'd had C++ background? We just need to provide resource which anybody capable and willing can use. That's it. It does not require a research to know that OOP developers will have to learn how to use traits instead of classes, automatic memory management background will make it harder to explain what stack allocation means and lifetime should capitalize of existing intuition of what variable scope is and develop from here. It also does not require a research to learn that many times developers can not comprehend error message compiler generates.
In my opinion there are several directions coordinated effort to impoove learnablility can take.
First one is documentation (of course). We need honest assessment, does The Book achieve its goals? And what is the success criteria for documentation anyway? I propose the following definition for documentation success: Given reasonably competent software developer, documentation will produce reasonably competent Rust developer. It means that after reading the documentation the person is capable of writing simple but useful and practical application.
This definition is important in Rust case because current book sweeps under the carpet so many complex matters that it creates false illusion that you understood rust whereas it is nowhere close to the truth. It is acceptable to skip and simplify during the learning but what is not acceptable is to hide blind spots because it creates frustration later on.
Rust specific is such that you can not write even very simple practical application without quite deep understanding of very complex matters, like lifetimes and how lifetimes are interacting with other features. How this project is going to deal with this? This is hard question. And I feel like initial direction is going after simple yet irrelevant questions like "which background makes it easier to learn rust".
Is documentation sufficient? Rust's traits are brilliant but little known to masses. Understanding traits is not hard, but developing working techniques takes time, trail and errors. Your OOP thinking will be in the way for long time. Should we admit that developing proper use of traits can not be achieved by OOP exposed developers without practical exercises? Should we create koans as essential part of Rust training? When to use dyn is also an intuition which comes with experience, despite concept itself being simple, can koans be helpful here too?
Generally speaking, what is the list of areas where documentation gives false sense of knowledge and koans is the recommended way of learning? Should koan path be incorporated into documentation? Saying something like: here is the explanation of the concept, but we are sure you will not understand how to use it until you complete those koans.
Documentation fragmentation is a problem too. Rust by example is great, but for student it increase time to learn when you have to scan multiple sources. Should we promote The Book to be central resource for comprehensive learning? Should it absorb best material from other sources? What's the strategy?
Next one is the compiler. Best example I've seen is Elm compiler. Error messages are so great that even in the most complex situations it becomes a tool of learning the language or a tool helping you to understand the structure of your program. I clearly remember I've learned some language features from compiler errors. And I remember reading Elm author saying that it was conscious effort requiring modifications to the compiler to carry relevant structures around to be able to produce good quality error messages. So just "let's be better at error messages" will not work. It requires designing compiler in certain way. I think I remember reading somebody from rust compiler team, saying similar things, that there is a dilemma, errors are created after de-sugaring, that's why compiler messages are cryptic, because at this stage compiler sees not what programmer sees. So, what is the strategy here?
In general, I can see a tendency for compilers to acquire more metaprogramming features. Great, it allows solving many problems in more elegant and performant way than runtime reflection. But the drawback is that metaprograming has no debugging no troubleshooting. With complex type system much more decisions are made by compiler at compile time, but compiler is not capable of communicating them in efficient way, if at all. Current compiler error handling is still rooted in "dragon book" approach (at least it feels this way). What is strategic direction here?
I'm under the impression that this potential learning project group would be an orthogonal effort to the compiler error and lint systems. However, I wanted to reply to your comments regarding the compiler, as I feel quite positive about both the current state and the future direction rustc is going within the context of learning/error reporting/code assistance.
Personally, rustc is the most helpful compiler I've ever used. It highlights what code segment gave it a problem, it tells you where it inferred it's expectations, it often tells you the solution or at least a possible solution. There are use cases it doesn't handle and the suggestions are not correct 100% of the time, but it's still much better than any other I have encountered. There are also markers for when the suggestions are machine-applicable or not and so on, which IDEs can take advantage of. And when I was still finding my feet with Rust, the errors and --explain texts certainly led me to learn about new areas of the language.
So I'm always curious and interested when I read a post like yours, when someone feels that rustc is coming up short. I'm genuinely unsure if my experience is just limited in this regard, and my expectations are too low. I'm not familiar with the Elm compiler, for example. Are there specific areas that rustc falls short in which other compilers are doing better? Are there some particular things that Elm does which rustc doesn't do which would be more helpful, for example?
Even if I do just lack helpful-compiler experience, though, I do feel that there is a conscious effort to (continue to) improve the compiler in terms of error handling and assistance.
Now, I am not on the compiler team (or any Rust team), so I can't truly speak to their strategies. But I see lots of patches/effort going in to the compiler to improve error reporting, including carrying additional data around to aid in the error messages; if you scan updates in This Week In Rust you can see plenty of them as well.
I've also read enough posts and watched enough presentations to know that the direction rustc is taking is query based instead of batch processing based, more language-server-protocol centic, and explicitly a model that has evolved beyond "dragon book design". This talk by Niko Matsakis particularly comes to mind. (Edit: Watched another video (same speaker, same conference, different talk) after looking up that reference which contains an even more relevant portion about "error messages are the first teacher". )
So I do think these are concerns which are actively being addressed in rustc. Are there any particular problems you've encountered, or other reasons that you feel otherwise?
I'm not sure if this was true at some point in the past, but it is definitely not true today. For example, the error message you get from moving into a for loop is different from the one you get if you de-sugar the loop.
The Rust compiler is specifically designed for delivering useful error messages, and a lot of effort every month goes into improving both individual error messages and the overall architecture.
Thanks everyone for the awesome additional feedback!
Coming up with clear ways for how people can participate will be one of the first orders of business.
This is very true. I agree that this should be one of, if not the very first thing we do.
The hope was that this group would define hypothesis around the topic of learnability of Rust and produce data that either supported or denied each hypothesis. This data could then be used by Rust educators or anyone else for whatever purpose they want to including improving how Rust is taught.
This was an unintentional mistake on my end as I (very wrongly) assumed educators would participate without an explicit call for participation. This is absolutely not the case. Having educators participate is essential.
This was not my intention. I do think that lessons learned can help with the language itself, particularly around diagnostics and or quality of life changes (e.g., things like the 2018 module system). I don't expect the data to provide revolutionary insights, but it might help teams prioritize or make better sense of anecdotal data they see/hear. i do think a majority of the insights will come into how Rust is taught, presented, etc. Some of this might impact the compiler and core tooling themselves but much of it will be aimed at educators.
I originally did not mention "learning styles" an received feedback that I should. I agree with your feedback that leaning on the idea of learning styles can be harmful. Perhaps rephrasing this as "learning preferences" would be helpful. This has more to do with ensuring that people can learn how they want to learn than it does with some innate "mode" each of us has for learning.
While I definitely think that we need to be careful about biases returned in self-reported data, I do think that survey data can provide interesting insights mainly into the perceived difficulty of learning Rust. I'm interested in learning how perceptions of Rust's learning difficulty change across different cohorts of learners. Survey data can be a reliable source of data on perceptions even if those perceptions don't necessarily match reality.
I both agree and disagree with you. Educators are a wealth of information that we should definitely tap into. I wasn't planning on directly trying to tap into their insights for the first activity, but it seems many believe we should do that first thing. I'm happy with getting insights from educators sooner rather than later.
However, I don't believe educators will be a superior source of information in every case. I myself am a Rust educator, and it was exactly because I don't have deeper insight into Rust that I wanted to create this project group. My only goal when teaching Rust is to teach Rust. I often don't have the time or ability to gather deeper insights into why people are struggling with certain topics. This group can explicitly take the time to understand why and how people are struggling to learn certain topics.
This is a fair point. Feedback from relevant stakeholders such as educators, Rust team members, etc. would be the first point of measuring success. If they don't find our efforts relevant, than we're not actually providing a good service. Second, part of the survey idea was to gather a quantifiable measure of Rust's learning curve. This can then be compared over time. While it's not necessarily the aim of this group to make Rust easier to learn, it is a goal to help others do so. So providing the information of whether Rust is becoming easier to learn (or at least perceived as such) is a direct goal.
I very much disagree with this assessment and the examples that follow it. Knowing how people's technical backgrounds impact how they learn Rust is extremely valuable. I already teach Rust differently when I know my audience is made up exclusively of C++ developers vs a general programming audience. Having better insights into how $LANGUAGE_X developers learn Rust can help me better tailor educational material for those developers.
Interestingly, I disagree that developers often can't comprehend error messages. This is direct evidence that "we" as the Rust community don't have a collective understanding of what makes Rust difficult to learn. Having data to directly back up what people do and do not struggle with would make conversations like this one much easier to have.
As @carols10cents already pointed out, the authors of the Rust book already receive enough feedback. The book is one of many resources for learning Rust. We shouldn't be focusing on one particular learning resource but rather focusing on the Rust learning experience holistically. Again, Rust educators can use this information to improve how they teach Rust, but we don't need a group to review specific learning artifacts.
I've made changes to the proposal which you can find here: A Potential Rust Learning Project Group - HackMD
OK, so this sounds like you want to set up experiments to test how different approaches help (or hinder) people in learning rust, which sounds like A/B testing. Is it possible to develop a course that more closely ties in to https://play.rust-lang.org/? The rough idea I have is the following:
For each concept:
The work that they complete, and the mistakes that they make along the way while trying to complete it, becomes data that can be used to evaluate different methods of teaching. Using this, we can have multiple parallel 'tracks' teaching the same concept, which can then be compared to one another to see how well students perform. Students can be assigned to the tracks at random, which should help us evaluate how well different methods work. Combined with a survey that they fill out before hand, we can figure out what works for different types of people (e.g., experienced programmers in C/C++ might have different needs from novices).
I hope it should be obvious that there are privacy implications with this method of testing. I would suggest that it be opt-in, with ways of engaging the students so that they can tell us what they had trouble understanding.
I came here to say the same thing. The jump from simplifications that get people over the first hump of getting rust working to understanding the complex parts of the language is a HUGE one. Many times I have only been able to do it with 1. talking about it with colleagues 2. rarer: reading amazing blog posts like your amazing one on lifetimes. 3. reading std code 4. rarest: reading rustc code 5. trying out examples in the playground.
I think a big problem people have is when doing fairly complex things with things like 'static, Sized, trait objects, or lifetimes, being able to PROVE which of these are true is extremely hard: 1. rustc is giving me this error cause I'm doing something actually bad 2. this is a wart and it should work but doesnt right now 3. I can do what I want, I'm just doing something slightly wrong.
Separately, I have some experience teaching rust, both intros and more intermediate/advanced problems where I work, and would love to see how I can help, will keep watching this thread to see what @rylev says about contribution to the working group!
@rylev would you be open to hosting office hours? I'd enjoy the chance to actually talk over Zoom/Discord/whatever with you and anyone else that wants to discuss the project charter.
Separately, I wanted to mention one point of focus -- APIs. This discussion + the charter are heavily focused on Rust-the-language as the thing to be learned and taught. But I hypothesize that most people learn Rust specifically to build applications in a certain domain: CLIs, web apps, data processing, embedded systems, so on. Hence, a newbie's experience with the language is filtered through the lens of the APIs they use.
For example, most people learning Python outside an "intro to CS" class do so in one of a few contexts:
Berkeley's Data8 curriculum is a good example of a curriculum that teaches a subset of Python in combination with a specific API.
Thinking about APIs is useful for two reasons.
Yes! This sounds good. Here's what I'll do:
This is a fantastic point. I think we'll want to try to come up with some criteria about which APIs are the right ones to focus on. This will unfortunately require us to play favorites in some case, but hopefully we can work around that somehow.
This fits very neatly into my idea of cohorts as well. I imagine a large percentage of Rust learners will have 1 (or maybe 2) specific domains that they will focus on. We can compare these cohorts against each other to see if there is currently a domain that allows for easier learning - which we can later investigate further.
I propose we meet on Tuesday (2021-01-12) at 17:00 UTC (18:00 CET/12:00 EST). We'll use Zoom since that seems to be the choice for most Rust related meetings. Please let me know if you would like to attend, and I can message you a link on this forum or over email.
The agenda will follow the structure of the proposal document I linked above. You can find the agenda here.
We will record the meeting and likely post it on YouTube. Of course, this means that if you join, you are ok with having at least your voice recorded should you wish to actively participate. If you have concerns about this, please let me know.
