Rust libz blitz!


#75

@cuviper I didn’t realize you had done so much on rayon. Can you join the libs meeting on 8/29, 1 PM PST?


#76

@brson Sure, I’ll be there, thanks!


#77

To account for our accomplishments at the end of the year, I’ve put together a spreadsheet tracking the contributions to both crates and the cookbook, by individual contributor. Note the pivot table sheets that sum up contributions.

So far it counts fixes to 99 crate issues, and 32 cookbook recipes. That’s pretty great!


#78

OK, I’ve taken the bold step of scheduling rand for 7/25 (which means we have to kick-start that evaluation next week).

I’m going to suggest that our disposition here with rand should be to polish it and release 1.0, limiting ambitious design changes. rand has been the subject of a lot of criticism over the years, and has suffered from not having a dedicated maintainer, but rand is also the best way we have to get random numbers today, and it’s just fine at that. There’s going to be a strong temptation to not settle on good enough, and demand a complete redesign, but the fact is that has not happened in the years people have been wanting it. There’s plenty we can do to improve the crate we actually have, and we should do that.

So let’s do a thorough review, and be cognizant of scoping the resultant work to what we can reasonably accomplish now, and what we can accomplish later.

Perhaps getting out a limited-ambition rand 1.0 will inspire someone to come along and design a better rand, and that can become rand 2.0, or it will deprecate rand. Let’s give it a shot and see what happens.

We need somebody to lead this evaluation. Anybody interested?

cc @bstrie I know you’ve talked about working on rand recently.


#79

I don’t think this is true at all. I spent significant amount of time documenting just a subset of the serious deficiencies with its design already: https://github.com/rust-lang/rust/issues/27703#issuecomment-178143920.

It’s already done better in ring.


#80

I’ve also triaged the list of “out-of-band” evaluations for crates that are ready to be evaluated. They are updated in the op and also listed here along with their maintainers.

These are important crates that for whatever reason the libs team is unlikely to dedicate one of their meetings to. But they still deserve the same polish and review that we pride ourselves on.

This list is composed of crates who’s authors have already assented to having their crates scrutinized as part of the libz blitz. I am additionally contacting the authors of a few others crates.

Volunteers are needed to lead these evaluations. If you are interested say so on this thread.

The process for leading an evaluation is roughly:

  • Add the crate to the OP of this thread, either to 'crate schedule’ if it’s going to a libs team meeting, or to ‘other crate evaluations’, for ‘out-of-band’ crate evalautions (like these).
  • Open an internals thread using the evaluation template, following the example of previous evaluations. Convert the op to a wiki with the wrench icon in the bottom right set of options.
  • Open an issue against rust-cookbook to brainstorm examples.
  • Copy the API guidelines checklist to an etherad, linked from the evaluation op.
  • Encourage contributors to fill out sections of the guidelines checklist, to make free-form observations on thread, and suggest cookbook examples.
  • As guidelines are completed, make notes in the op about potential issues to file against the crate once the evaluation is complete, and matters that need more discussion.
  • Work with the crate author to note pre-existing issues that block a 1.0 release.
  • After a few weeks, organize the issues and discussion points. Work with the author and community to agree on a set of resolutions and file them as issues on the crate repository, linked to a single tracking issue. Follow existing examples.
  • Periodically follow up on the cookbook and crate tracking issues to help drive them to completion.

Hm, this might sound like a big task, and it kinda is, but it’s important work, and so far has resulted in valuable improvements and contributions.


#81

@briansmith ring is indeed a great crate, and it’s on the candidate list. I’m glad to hear you are confident that its random number facilities are the best in the ecosystem. It’s been on my shortlist of crates the libs team might want to schedule soon.

Are you interested in having the community / libs team evaluate it as part of the libz blitz sometime?


#82

@briansmith How close do you think ring is to a 1.0 release? Might it happen this year?

Literally releasing 1.0 isn’t necessarily a goal here, but it is a consideration.


#83

I think it would make more sense to stabilize a new version of rand with a much simpler API like ring’s and the implementation details from ring (e.g. the feature flag to turn off the fallback to /dev/urandom for facilitating sandboxing that blocks file I/O syscalls).

There are lots of things that you could do in libstd that could be done in Rust, which currently require C/asm in ring because of the lack of stable (proper) access to syscalls and RDRAND/RDSEED in stable Rust, which is what ring targets.

In other words, I’m not proposing that ring be the replacement for rand, but rand’s API should be fixed to be closer to ring’s than its current API.


#84

1.0 is partly marketing and partly about some kind of commitment to a stable API.

As far as marketing goes, I’ve debated releasing 1.0, 2.0, 3.0, etc. instead of 0.6, 0.7, 0.8, etc. but it doesn’t matter enough for me to bother with.

As far as the API stability goes, I know there will be more breaking changes (e.g. the AEAD API) this summer (I hope) but I’ve not considered trying to finalize any part of the API. Especially there are some parts of the API that would change if/when things like const fn and statics in constant position (e.g. x: [u8 ; ring::digest::SHA256::output_len];) were/are added to the language.

Regarding some things that are in the API guidelines, like the capitalization requirements for SHA256 vs Sha256, I’d like to see some evidence showing that those things actually have a significant positive difference vs. being over- or wrongly- specified before I make changes like that.


#85

@brson I’m happy to pick up same-file, since we’ve already been over it a bit in the walkdir evaluation


#86

Awesome. It’s yours! You can just run it in the same way as the previous one.

This week I am at a Mozilla all-hands and won’t have much time to devote directly to this, but next I’ll make sure we get some more evaluations underway.


#87

Thanks for the feedback @briansmith. I’ll try to incorporate it into the rand evaluation, and to solicit more of your advice when it gets underway.


#88

I’ve started a topic for same-file.

I thought it was still worth time-boxing the evaluation component even though there isn’t a libs meeting. I gave it 2 weeks from today, but am happy to change or remove that.


#89

As we are quite low on actionable example ideas in cookbook. I was thinking about adding some:

But some crates have pretty awesome tutorials/cookbooks by them selves like csv does. What would be the approach for such crates?


#90

Looks like I haven’t got permission to make the same-file OP a wiki. No wrench icon produced by expanding the ... for me :disappointed:


#91

I don’t pretend to know the inner workings of Discourse, but I used my powers to make it a wiki. :slight_smile:


#92

I think we can steal them, with the authors’ permissions!


#93

Thanks @KodrAus!

@budziq, yes please, begin creating more recipe ideas.


#94

Argh, it looks like I missed the status update last week again, and I’m not going to get it done today. Sorry for falling behind, but I will catch up next week. Thank you so much @KodrAus, @budziq, and @burntsushi for keeping things moving :heart:

Next week I will endeavor to catch up on reviews, open the rand evaluation, resolve a number of guidelines issues, and close out some of the early evaluations with version bumps and releases.

It came to my attention recently that CC0 would be more a more appropriate license for the cookbook, so I filed an issue against it: https://github.com/brson/rust-cookbook/issues/209