Production user research summary

[phildawes]

Thanks for doing this @brson, always good to get feedback.

Racer is still too slow to be practical

This surprises me (racer author). I would have thought the complaint would be ‘racer is not complete enough’ rather than performance. Performance on a completion on my linux laptop is ~100ms - i.e. the same sort of speed as intellij.

Maybe there’s some environment related performance issue I’m missing - e.g. perhaps racer is really slow on windows or when used with a particular editor or something. Did you get a feel on what their environment was like?

Thanks!

[skade]

Since I’m alone, do this on my spare time, and don’t raise any money from it, I don’t have enough weight to be in these meetings (it’s not a complaint, I don’t have any problem with this).

I think this is a misconception. “production users” in this case was meant in the sense of “people that build projects with the intention of releasing things using Rust”. It’s very loosely defined. I wasn’t involved in the planning, but to my impression, the group of people present was more defined by people the core team knew were using Rust in such a fashion/intention. Making money is also no criterion. It’s not meant as an exclusionary attribute. It’s just the meeting where advanced issues that might pop up in actual project use later are discussed, as opposed to meetings were problems with beginner teaching are discussed.

This won’t be the last of such meetings and your feedback is very valuable and welcome. Would you mind if I get you in touch with the core team?

Florian (Rust community team)

[tomaka]

This won’t be the last of such meetings and your feedback is very valuable and welcome. Would you mind if I get you in touch with the core team?

Sure!

[thijsc]

We’re working on multiple projects using Rust at https://appsignal.com, ping me at thijs at appsignal.com if you would like us to join next time.

[nikomatsakis]

We’ve heard some anecdotes that there is no mode to build Rust that is both reasonably fast to build (ala --opt-level=0) and also fast enough to run.

It’s worth mentioning that @alexcrichton did some experiments and found that -O1 with codegen-units had almost the same compile time as -O0 but almost the same performance as -O3. We should make this an easier option to choose, I think.

[tomaka]

It’s worth mentioning that @alexcrichton did some experiments and found that -O1 with codegen-units had almost the same compile time as -O0 but almost the same performance as -O3. We should make this an easier option to choose, I think.

The image library definitely runs much faster with -O3 than with -O1.

With the code I’m working on, loading the assets at runtime takes ~10.5 seconds with -O1 and ~1 second with -O3

[nikomatsakis]

@tomaka interesting. I didn’t mean to imply we should stop using -O3, however, just that for many users it may be a useful choice.

[excaliburHisSheath]

I can’t claim to have the same depth of experience as @tomaka, but I’m in the same spot of wanting to build and release games using Rust. If there’s anything I can do to contribute let me know.

[aturon]

-O1 with codegen-units had almost the same compile time as -O0 but almost the same performance as -O3

Did you mean the same performance as -O1?

[nikomatsakis]

@aturon no, I meant what I wrote. Keep in mind, this test was just cargo, afaik, so it’s unclear how representative it is.

[aturon]

@aturon no, I meant what I wrote. Keep in mind, this test was just cargo, afaik, so it’s unclear how representative it is.

FWIW, I just checked with @alexcrichton and he says it’s about 3-5x faster than -O0, which is still much slower than -O3.

[nikomatsakis]

FWIW, I just checked with @alexcrichton and he says it’s about 3-5x faster than -O0, which is still much slower than -O3.

OK, yes I misremembered. Here were the results @alexcrichton sent me :

• O0 - 1:01 build time, 1.5s runtime
• O1 - 1:09 build time, 0.5s runtime
• O3 - 2:13 build time, 0.17s runtime

still, much closer to -O3 than -O0.

[brson]

@tomaka Thank you for the feedback. The SipHasher issue is a persistent gotcha, and seemingly so easy to solve (both by having a weaker hash available and by improving our strong hash).

@achanda I’d love somebody to work on this! My present opinion is that Crater should expose information via API for crates.io to use, about both observed platform and channel compatibility. Crater seems the best option because the infrastructure is set up (though it’s a mess and doesn’t provide all the stuff we need) and will be maintained as long as I work on Rust. I’m interested in hearing other designs though.

@gmjosack Thank you for the feedback about Cargo. I totally agree this issue of running Cargo independently from crates.io (and the network at all) are major blockers for a lot of use cases (including Gecko’s). Personally, I think it’s important enough that it should be on my shortlist of ‘showstoppers’ that need attention soon. cc @larsberg @alexcrichton.

[larsberg]

The Cargo-specific issues I can find from Gecko (compiled from bugzilla threads and the Portland meeting notes):

• Desire to compile multiple toplevel static libs concurrently, but Cargo doesn’t support concurrent access.
• Want to unroll all of the dependency tree in-tree with manual upgrade / pick-up of those external deps (called “vendoring” in FFOS parlance). This doesn’t mesh well with Cargo’s desire to own the place where dependencies live unless you .cargo/config paths override them all.
• Absolutely totally unacceptable to go online “automatically” during a build. Should be an error if something needs to be updated/refreshed from the network.
• Artifact caching support & deterministic builds via it (though not required at day 1).

[nrc]

This (internet access for Cargo) is also a bit of a show-stopper for some universities which have restricted internet access (the student I’m supervising can’t use Cargo at uni, for example).

[achanda]

@brson @alexcrichton I will start another thread on the platforms/rustc version thing.

Re: prod users not using cargo, is this because there are no guides on running an internal version of the crates.io (or the index)? I’m not sure if that’s even possible. If not, we should consider expanding on it. I’m thinking something like the private docker registry.

[brson]

@thijsc Thanks for speaking up! I’m making a note to myself to include you next time we do something like this.

[gmjosack]

Re: prod users not using cargo, is this because there are no guides on running an internal version of the crates.io (or the index)? I’m not sure if that’s even possible. If not, we should consider expanding on it.

I have a draft of such a post that I need to finish up, though I’ve been busy with a “Hack Week” at work this week so haven’t been able to finish it. Hopefully should have a guide on setting up an internal mirror sometime this weekend.

[achanda]

Awesome! Looking forward to it.

[bvssvni]

Some input:

• The /r/rust_gamedev subreddit has stayed at 10% of /r/rust for a long time, which indicates that about 1/10 people looking into Rust are also interested in this domain.
• The language is pretty good now for the simple stuff, and the ecosystem for gamedev is evolving rapidly.
• Performance is a major engineering task in gamedev, so addressing this is going take lots of discussion and details. I think this needs to be solved by addressing the individual problems. Talk to @tomaka, @dobkeratops, @kvark, @csherratt, @cmr , they know this stuff.
• The Piston project is stressing Cargo to its limits, one problem was a stack overflow bug, but I think it was fixed/working on now. Unfortunately this bug made it into stable.
• There was a problem with examples that uses libraries that depends on itself, but we solved this by moving the examples to another repo. Not sure if anything can be done, though.
• Biggest pain point: Dependency conflicts!!! There should be an error message explaining that it tries to link two different versions of the same library.
• Great to see tools like Crater and Cargo-Crusader!
• The goals for 2016 looks good. Specialization is a big one.
• The guidelines for get/set properties separates Builders from normal objects, while in practice we have found lots of use cases where there does not have to be a sharp distinction. I believe I’ve missed the RFC. Perhaps prefixing with “get_” is better when there is a non-readonly property, such that a builder method can be added later if desirable. The pattern we are using has worked well: https://github.com/PistonDevelopers/piston/issues/882.
• Many users think of lifetimes on structs as a “design pattern”, while in practice it is used for optimization. For structuring larger programs, Rc/Cell/RefCell/Arc could be encouraged and highlighted for people that come from GC languages like Java or C#.
• Since I have little time to follow the RFC process, I am in grateful for the “final comments period” section on TWiR.
• There is a case where Rust fails at reasoning about equality for associated types, which blocks a pattern that might simplify generic libraries https://github.com/PistonDevelopers/backend/issues/16. This is somewhat special case, so it is not high priority.
• The num library could be split up into smaller crates, in particular traits that you often want to use without needing the rest of it. In order to make things more stabilize in Piston we moved away from it in some libraries at the bottom of the dependency graph.
• I’m very busy using Rust already and overall the language is pretty good. The major challenge I see now is scaling up projects, which I think will be solved by more tools. The core team does a great job!
• There were plans about “Redox”, a game competition web site for Rust that might bring more attention. Now that the web ecosystem is improving perhaps we’ll make another shot at it.
• The gamedev ecosystem is going to be excellent in a few years. We are aiming high and lots of people are contributing. Since this is so large field, the people are stretched across many projects. I expect this to improve over time.