[Pre-RFC] Allow for Analysis of Rust Programmers logic (Rust_Language_Analysis)

deg4uss3r · 2017-03-05T19:19:29.111Z

I’m on IRC as degausser in both freenode and mozilla. Or just drop me a line via github.

Content located here:

deg4uss3r/rfcs/blob/master/0000-rust-users-langauge-analysis.md

- Feature Name: Rust_Language_Analysis
- Start Date:   2017-03-05
- RFC PR:       
- Rust Issue:   

# Summary
[summary]: #summary

As the [Rust](https://www.rust-lang.org) language evolves, it's important to continue to implement new features and use them benefit the entire community of Rust-lang programmers. This means we need to understand:
* _how_ developers are using the language
* _what_ they normally struggle with
* _what_ can give the Rust community the biggest benefits either as a future RFC or just an additional chapter to the [Rust book](https://doc.rust-lang.org/book/) or [Rust By Example](http://rustbyexample.com/).

# Motivation
[motivation]: #motivation

The motivation for this RFC was Rust's [blog post on Ergonomics](https://blog.rust-lang.org/2017/03/02/lang-ergonomics.html) and several comments in the [Hacker News comment chain](https://news.ycombinator.com/item?id=13785277) specifically comments made by [modeless](https://news.ycombinator.com/user?id=modeless), [d33](https://news.ycombinator.com/user?id=d33), and of course [shepmaster](https://news.ycombinator.com/user?id=shepmaster). Rust appears to be at an impasse with what to formally implement and how to make the language better for both new and old users; therefore, a formal verification on how users _use_ Rust is necessary to make informed decisions.

# Detailed design
[design]: #detailed-design

This file has been truncated. show original

konstin · 2017-03-05T20:14:35.162Z

Instead of using the rust playground, an analysis tool like proposed here should be hooked onto crater. This has the advantage of using real world applications instead of dummy code, getting metadata such as the popularity of project and getting all the information directly from rustc as a by-product. Having collected such data, it could e.g. be used for correlating a project’s popularity or age with the features and design patterns used and for arguing about the real world usage for proposed RFCs.

Another approach would be publishing a cargo plugin collecting error types and other stats which will then be anonymously reported. The problem here is that most people don’t like having tracking tools on their machines.

withoutboats · 2017-03-05T22:31:45.496Z

This RFC is missing some important information. It never seems to explain what is being added to playground to enable this analysis. Presumably some sort of collection of inputs (just submitted program files? or each key press?). Which will then be analyzed how? It never seems to say.

@konstin crater analysis can’t tell you what errors people make because (at least in theory) those crates are all what someone got to compile and then released. However that data would be useful in other ways.

konstin · 2017-03-05T22:45:39.443Z

I meant that the cargo plugin would be published to the developers where it would do analytics stuff in the background while they’re doing their code - compile - fix errors cycle . I’m not really in favor of that, but it’s a possible option use by some other projects.

oli-obk · 2017-03-06T12:29:07.771Z

I have actually been doing some research into this for a while now, since I want to automatically detect patterns that could be linted against or where the error/warning could produce a suggestion that does what 90% of use cases want.

Maybe as a RLS extension, since that would contain much more detail.

deg4uss3r · 2017-03-06T18:55:38.673Z

I think this is also a good idea. Having something like --send-diagnositcs would be great or to specify it also in ~/.cargo/config for a default way to always send them.

deg4uss3r · 2017-03-06T19:11:56.576Z

Thanks, I agree. I did leave some things out on purpose to get the communities reaction to this general idea (also why it is a pre-RFC :)).

To answer some of the initial questions:

I did specify (though very subtlety) in the draft that I think keylogging on the playground would be beneficial to understand what trips up programmers working within the playground. Taking interval snapshots would be also useful and a little less invasive. Also option is basically what @konstin and @ker were eluding to as your developing a --send-diagnostics would be great to have as an option every time you run cargo build.
As for what data is collected I also wanted to open that up to discussion since I am not a language expert to know what is truly beneficial. I was thinking along the lines of the follow data:
any time a user runs Play record the code submitted
any time a complication is made record the result (especially any compiler error messages)
record or give the user any options to submit and resources used
- Possibly record the way the user interacts with docs.rust-lang using cookies, but this might be a little too privacy invasive
record session time (e.g. how long as the window been open without the text input cleared)
record lines that have changed (plus timing information)

That’s all I can think of at the moment on the data capturing. Like I said, I am open to ideas, but I really think this could benefit and give us real data to help shape the language the way it most makes sense. Does this make sense now? I will update the RFC in the git with this information this evening.

system · 2019-03-25T08:28:07.940Z

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.