Rust libz blitz!

[brson]

I added rayon and mime to on-deck evaluations. rayon is about ready for 1.0, and very popular. mime also gets around a lot and libs team has expressed interest. I demoted env_logger to out-of-band evaluations, as I don’t see the libs team revisiting logging any time soon. I’m still hopeful to get some such evaluations going soon.

The csv crate is also about ready for 1.0, and would be worthy of review. As a @burntsushi crate it’s going to be super high quality. I’m a little iffy about whether it is within the present mandate of this project, though certainly csv is a common format, and is sometimes supported by standard libraries.

[burntsushi]

RE csv: It’s at 1.0.0-beta.3 right now, which was intentional so that I could still make breaking changes. My plan is to probably get 1.0 out before the end of the year. I agree with your concerns @brson though, but I just wanted to put this out there!

[budziq]

I would like to mention that @kbknapp has started a libz Blitz Style overhaul for clap-rs some time ago

[KodrAus]

@brson Were we adding same_file as an out-of-band crate?

I’m happy to kick that off next week, it shouldn’t block walkdir since we’re removing the re-export.

[brson]

@KodrAus Yes let’s add same_file to out-of-band.

@budziq oh cool, thanks for the heads-up!

I’m just going through and triaging stuff and realized I did not post a status update last week! Sorry about that. I will do two weeks in the next report.

[brson]

Rust Libz Blitz status update 2017-06-15

Ok, this is an update of the last two weeks.

The big event this time was that @KodrAus finished the walkdir evaluation, and it went really well. He filed a bunch of very well-documented issues that are all still open and want to be closed.

The error-chain evaluation is open too and in need of help completing the guidelines checklist and brainstorming ideas for cookbook recipes. There’s also a fair bit of discussion on that thread about the error-chain design. If you have opinions don’t hesitate to jump in.

Coming up we’ll expect @burntsushi to get the gcc evaluation started soon, and for me to schedule out the next few crates after that.

I also hope to start a push for getting actual 1.0 crates released, and to motivate that with a “1.0 achievements” thread of some kind, where any crate author can post their crate that meets some qualifications to the best of their determination.

It was another really productive week. Thanks to everybody who chipped in!

url

• @SimonSapin updated the percent_encoding docs
• @SimonSapin moved percent_encoding to its own crate
• @tomecki made HostAndPort impl Display
• @imor implemented Debug for many types
• @cGuille implemented Hash for Origin
• @SimonSapin hid the quirks module
• @Enet24 replaced unwrap with ? in examples
• @tmccombs explained ‘fragments’

log

• @budziq added doc references to other loggers
• @budziq added docs for interaction between features and log!
• @rap2hpoutre improved max_log_level docs
• @nivkner improved set_logger_raw docs

reqwest

• @budziq exposed RedirectAction and RedirectAttempt
• @tomprince hid internal conversions
• @tomprince made Error::get_ref return Send+Sync
• @theduke moved try_ macro to error.rs
• @alisha17 converted examples to ?
• @budziq added “Errors” sections to docs
• @jaemk improved Response docs
• @jaemk added examples to Body ctors
• @Andygauge improved error docs
• @jaemk improved RequestBuilder docs

cookbook

• @budziq added “compress a directory into a tarball”
• @sb89 added “extract all links from a webpage”
• @budziq added “define and operate on a type represented as a bitfield”
• @budziq upgraded rayon and image
• @budziq pinned mdbook to 0.0.22
• @budziq clarified the cookbook license
• @crypto-universe added “recursively calculate file sizes at a given depth”

help wanted

• walkdir: Add Error docs to methods that return Result
• walkdir: Use ? in docs instead of unwrapping
• walkdir: Add example for content_first
• walkdir: Add links to other walkdir items in DirEntry docs
• walkdir: Add links to other walkdir items in Iter and IterFilterEntry docs
• walkdir: Add links to other walkdir items in WalkDir docs
• walkdir: Add links to other walkdir items in WalkDirIterator docs
• walkdir: Document that Iter and IterFilterEntry are the result of trait methods
• walkdir: Correct errors in WalkDir type docs
• walkdir: Add html_root_url attribute
• walkdir: Implement Debug for WalkDir, Iter and IterFilterEntry
• walkdir: Add build badges to Cargo.toml
• walkdir: Add categories to Cargo.toml
• walkdir: Rename Iter to IntoIter
• walkdir: Rename IterFilterEntry to FilterEntry
• walkdir: Link references to std in docs
• walkdir: Make skip_current_dir and filter_entry inherent methods
• walkdir: Make WalkDir Send + Sync
• walkdir: Document why unwraps won’t fail
• walkdir: Remove re-export of is_same_file
• walkdir: Change OsString args in sort_by to OsStr
• walkdir: Move DirEntry::ino method to an extension trait
• cookbook: Use filter_entry in walkdir example
• cookbook: “decompress a tarball while removing a prefix from the paths”
• cookbook: Move to skeptic 0.10
• skeptic (used by cookbook): multiple matching crates when deps are duplicated

[brson]

I updated the evaluation template to reflect the format used in the latest evaluation.

[brson]

I’ve scheduled rayon for 8/29 cc @nikomatsakis

That leaves 3 more slots for us to fill to fulfill our scheduled 16 crates for the year. I am not sure yet what will happen schedule-wise after that, but probably the focus will be on actually getting those 1.0 version bumps so we have something to show for ourselves at the end of the year. I think though that we will probably maintain some kind of steady schedule. We’ll see.

In the meantime, I’m also looking at more crates for the “out-of-band” crate selection, that is crates we can run evaluations on at any time, but that the libs team is not going to conduct an entire meeting about. I’ll fill out that list a bit, and contact authors to get consent to review their crates.

[SimonSapin]

• @SimonSapin updated the percent_encoding docs
• @SimonSapin moved percent_encoding to its own crate

These were rebases of PRs by @AndyGauge and @seanmonstar.

[nikomatsakis]

cc @cuviper

[brson]

@cuviper I didn’t realize you had done so much on rayon. Can you join the libs meeting on 8/29, 1 PM PST?

[cuviper]

@brson Sure, I’ll be there, thanks!

[brson]

To account for our accomplishments at the end of the year, I’ve put together a spreadsheet tracking the contributions to both crates and the cookbook, by individual contributor. Note the pivot table sheets that sum up contributions.

So far it counts fixes to 99 crate issues, and 32 cookbook recipes. That’s pretty great!

[brson]

OK, I’ve taken the bold step of scheduling rand for 7/25 (which means we have to kick-start that evaluation next week).

I’m going to suggest that our disposition here with rand should be to polish it and release 1.0, limiting ambitious design changes. rand has been the subject of a lot of criticism over the years, and has suffered from not having a dedicated maintainer, but rand is also the best way we have to get random numbers today, and it’s just fine at that. There’s going to be a strong temptation to not settle on good enough, and demand a complete redesign, but the fact is that has not happened in the years people have been wanting it. There’s plenty we can do to improve the crate we actually have, and we should do that.

So let’s do a thorough review, and be cognizant of scoping the resultant work to what we can reasonably accomplish now, and what we can accomplish later.

Perhaps getting out a limited-ambition rand 1.0 will inspire someone to come along and design a better rand, and that can become rand 2.0, or it will deprecate rand. Let’s give it a shot and see what happens.

We need somebody to lead this evaluation. Anybody interested?

cc @bstrie I know you’ve talked about working on rand recently.

[briansmith]

rand is also the best way we have to get random numbers today

I don’t think this is true at all. I spent significant amount of time documenting just a subset of the serious deficiencies with its design already: https://github.com/rust-lang/rust/issues/27703#issuecomment-178143920.

Perhaps getting out a limited-ambition rand 1.0 will inspire someone to come along and design a better rand, and that can become rand 2.0, or it will deprecate rand. Let’s give it a shot and see what happens.

It’s already done better in ring.

[brson]

I’ve also triaged the list of “out-of-band” evaluations for crates that are ready to be evaluated. They are updated in the op and also listed here along with their maintainers.

• cfg-if - acrichto
• csv - BurntSushi
• mime - seanmonstar
• regex - burntsushi
• same-file - BurntSushi
• semver - steveklabnik
• tar - acrichto
• threadpool - frewsxcv
• unicode-segmentation - mbrubeck, Manishearth, SimonSapin, co-maintained

These are important crates that for whatever reason the libs team is unlikely to dedicate one of their meetings to. But they still deserve the same polish and review that we pride ourselves on.

This list is composed of crates who’s authors have already assented to having their crates scrutinized as part of the libz blitz. I am additionally contacting the authors of a few others crates.

Volunteers are needed to lead these evaluations. If you are interested say so on this thread.

The process for leading an evaluation is roughly:

• Add the crate to the OP of this thread, either to 'crate schedule’ if it’s going to a libs team meeting, or to ‘other crate evaluations’, for ‘out-of-band’ crate evalautions (like these).
• Open an internals thread using the evaluation template, following the example of previous evaluations. Convert the op to a wiki with the wrench icon in the bottom right set of options.
• Open an issue against rust-cookbook to brainstorm examples.
• Copy the API guidelines checklist to an etherad, linked from the evaluation op.
• Encourage contributors to fill out sections of the guidelines checklist, to make free-form observations on thread, and suggest cookbook examples.
• As guidelines are completed, make notes in the op about potential issues to file against the crate once the evaluation is complete, and matters that need more discussion.
• Work with the crate author to note pre-existing issues that block a 1.0 release.
• After a few weeks, organize the issues and discussion points. Work with the author and community to agree on a set of resolutions and file them as issues on the crate repository, linked to a single tracking issue. Follow existing examples.
• Periodically follow up on the cookbook and crate tracking issues to help drive them to completion.

Hm, this might sound like a big task, and it kinda is, but it’s important work, and so far has resulted in valuable improvements and contributions.

[brson]

It’s already done better in ring.

@briansmith ring is indeed a great crate, and it’s on the candidate list. I’m glad to hear you are confident that its random number facilities are the best in the ecosystem. It’s been on my shortlist of crates the libs team might want to schedule soon.

Are you interested in having the community / libs team evaluate it as part of the libz blitz sometime?

[brson]

@briansmith How close do you think ring is to a 1.0 release? Might it happen this year?

Literally releasing 1.0 isn’t necessarily a goal here, but it is a consideration.

[briansmith]

I think it would make more sense to stabilize a new version of rand with a much simpler API like ring’s and the implementation details from ring (e.g. the feature flag to turn off the fallback to /dev/urandom for facilitating sandboxing that blocks file I/O syscalls).

There are lots of things that you could do in libstd that could be done in Rust, which currently require C/asm in ring because of the lack of stable (proper) access to syscalls and RDRAND/RDSEED in stable Rust, which is what ring targets.

In other words, I’m not proposing that ring be the replacement for rand, but rand’s API should be fixed to be closer to ring’s than its current API.

[briansmith]

1.0 is partly marketing and partly about some kind of commitment to a stable API.

As far as marketing goes, I’ve debated releasing 1.0, 2.0, 3.0, etc. instead of 0.6, 0.7, 0.8, etc. but it doesn’t matter enough for me to bother with.

As far as the API stability goes, I know there will be more breaking changes (e.g. the AEAD API) this summer (I hope) but I’ve not considered trying to finalize any part of the API. Especially there are some parts of the API that would change if/when things like const fn and statics in constant position (e.g. x: [u8 ; ring::digest::SHA256::output_len];) were/are added to the language.

Regarding some things that are in the API guidelines, like the capitalization requirements for SHA256 vs Sha256, I’d like to see some evidence showing that those things actually have a significant positive difference vs. being over- or wrongly- specified before I make changes like that.