Rust libz blitz!

libs
1

The Libz Blitz is the library team’s major 2017 initiative: raising a solid core of the Rust crate ecosystem to a consistent level of completeness and quality.

Join the Working group Gitter chat and say hi, talk about the blitz and ask where help is needed.

Read the Working group doc to see how we’re tracking and where we need help.

I want to:

Lead an evaluation

See crates that need a lead then reach out on gitter

  • Help steer the direction of fundamental crates in the Rust ecosystem
  • Engage with experienced Rust contributors
  • Solicit feedback from the crate author, users and other community members

Contribute to an open evaluation

See open evaluations then reach out on gitter

  • Share your experience using a crate
  • Help fill in the guidelines checklist
  • Help think of examples for the cookbook

Contribute to an evaluated crate

See crates that need contributors and crates that need design work then reach out on gitter

  • Help fix beginner-friendly issues in evaluated crates
  • Help fix more complex issues in evaluated crates
  • Contribute your thoughts to bigger design questions

About the blitz

Let’s apply what we’ve learned in stabilizing std to raise a solid core of the Rust crate ecosystem to a consistent level of completeness and quality, and accomplish a number of other important objectives along the way.

This is a multi-faceted project to achive a number of inter-related goals around the quality of the Rust library ecosystem. It is intended to be amenable to contributions from both experienced Rust developers, and completely new Rust developers.

For more background, see the blog post.

This post is a wiki. Feel free to edit it.

Goals

The basic goal for 2017 is to get 16 key crates to “1.0” quality levels.

But there are several other goals here:

  • Raise the quality of crates in the Rust ecosystem to consistent high levels
  • Release 1.0 versions of crates
  • Produce API guidelines that capture established best practices for Rust crate design
  • Produce a cookbook of idiomatic examples of Rust crate usage
  • Consistently generate well-defined tasks for contributors who are looking to make an impact on important areas of the project
  • Demonstrate a reproducible process for evaluating crate quality, that hopefully other crate authors will appreciate and use.
  • Foster greater collaboration between the Rust libs team and the wider community of crate authors

At the end of 2017 we will aim to have a blog post summarizing the progress made in the Rust crate ecosystem this year, and to the first versions of the guidelines and the cookbook.

Who makes the blitz work

Crate lead

Each crate needs a volunteer to lead the evaluation effort. That entails starting up a thread, breaking up the evaluation work into small work items that can be taken on by others in the community, keeping the discussion moving in productive directions, making sure the evaluation is completed on time, presenting the results at the libs team meeting, and, finally, filing discrete, actionable issues on everything raised, and funneling them to TWiR.

Anyone can be a crate lead, but it’s a substantial commitment and is largely about organization, communication, and consensus, and requires presenting to the libs team in a video meeting. Say so on this thread if you are interested in leading a particular crate.

Crate evaluator

This is the preliminary work of comparing a crate to the API guidelines, identifying deficiencies, and raising observations about API design that may impact the guidelines. It will often require working with the crate author to understand a crate’s known shortcomings.

Each evaluation will be published to the coordination thread, providing an opportunity for general feedback. Open feedback allows for a wide range of voices to be heard and is an important check against design myopia.

This effort is collaborative and everyone is welcome to participate in this way at any time. The crate lead will help create opportunities to jump in.

Documentation slinger

A lot of the work involved in getting a crate up to full quality is improving documentation, including writing up cookbook entries. There will be lots of opportunities for this kind of high value work.

Everyone is welcome to participate in this way at any time. The crate lead will help create opportunities to jump in.

Library hacker

Somebody must do the programming work of resolving the issues on the roadmap to stability for each crate. We expect to produce many discrete work items and that many of them will be accessible to inexperienced Rust programmers.

Everyone is welcome to participate in this way at any time. The crate lead will help create opportunities to jump in.

Library designer

There remain some important libraries that are only lightly maintained but are known to require significant design work (again thinking of rand especially). These will not improve without a dedicated and skilled author to drive them forward. We are going to be looking out for authors with the design skills necessary to fix the problems, and the social skills necessary to navigate the process.

We will occasionally make pleas for help on specific design matters as they arise.

Libs team guest

The library team will spend one of their video meetings reviewing each crate evaluation. We hope that the crate authors will be interested in joining us, sharing their unique expertise, and getting to know the libs team. This kind of interaction creates strong bonds in a way that communicating through text does not. We hope this will foster shared values across the ecosystem, and pave the way for expanding the libs team more formally.

This will be by invitation and focused on authors with an existing reputation for high quality work and productive collaboration.

Crate author

The libs team has some specific functionality and crates it wants to focus on this year. Outside of that, we are hopeful that the process and guidelines we develop will be widely useful and that crate authors will independently seek to evaluate and improve their own crates in similar ways.

How we’re tracking

This is the schedule the libs team is keeping, and reflects the crates they have already reviewed, and the crates they desire to review in the near future.

We are only evaluating crates with the permission of their authors. These crates tend to be drawn from the crate listlower in this thread. Other past evaluations not reviewed by the full libs team are in another list.

Expand the section below for details on each evaluated crate.

Candidate crates

These candidates provide features that would be mostly included in a “batteries included” Rust standard library. For our limited purposes we can exclude many domains that are important but further on the periphery of what is needed for a general computing platform’s standard library (specifically thinking about multimedia, web frameworks, specific databases, and such).

At present there is no defined selection process, but the above criteria are a guideline.

We are only going to do evaluations of crates where the author agrees. This is a lengthy, critical process, that imposes new work on the crate’s maintaners, and it is reasonable not to want to be involved.

The working list: https://public.etherpad-mozilla.org/p/rust-crate-list

This list is only on an etherpad because Discourse seems to not support tables in markdown.

Please discuss candidates on thread before adding them to the list.

Lists of crates:

6 Likes
2

On HN mintplant suggested the bytes crate as an important building block. It does look promising and I know it’s used pretty widely. Seems like it might have some overlapping functionality with byteorder, but I’m not too familiar.

3 Likes
3

Hi, this is absolutely great! This will help raising up the consistency and developing experience within Rust ecosystem.

I’d like to add some more candidates.

1 . First, Data Structures

i’d like to see some more complex fundamental data structures get some solid reference implementations.

  • (something missing here) (for trees and forests)
  • petgraph and daggy (for graphs)

I’d love to see something like these give widely recognized and consistent ways of defining and organizing non linear-table-like ownership relationships between components.

And for the linear-table types:

can be used in quite many different situations.

2 . Numerical Types and Algorithms,

there’s already num and ndarray in the list, i’d like to suggest adding matrix too. Actually i think typenum is also very important before pi-types come…

3 . digesting, compression and codec algorithms

there’s already flate2 in the list, i’d like to suggest adding ring, lzw, and image. Surprised there’s not yet binding for libjpeg-turbo…

4 . utilities

i’d like to suggest adding gc to the list.

3 Likes
4

This is fantastic! Once again the Rust team happily surprises us with the breakneck pace at which it evolves the language ecosystem.

I’m wondering, since I would enjoy helping with the Rust blitz, who is appointing the different roles for each crate improvement effort?

Also, and more importantly, I think it would be neat to mark a list of issues in each crate as E-Easy to attract new contributors to participate in these core crates. For many it sounds like a daunting and intimidating task, but there is no wizardry involved. Having a mentoring guiding hand could help give confidence to newcomers to realize they can do more than they think. The more people contributing to core crates the merrier for Rust as a whole.

From my talks with a bunch of community members it is clear that Rust has done a great job at making “Systems programming” non-scary. Liberating many high-level programmers about how much wonderful stuff they can do.

Sign me up :smile:

2 Likes
5

Indeed it does not. Hopefully this will change once we move to CommonMark in v1.9.

For now we do however support vanilla HTML tables like this:

<table>
<thead>
<tr>
<th>foo</th>
<th>bar</th>
</tr>
</thead>
<tbody>
<tr>
<td>baz</td>
<td>bim</td>
</tr></tbody></table>

You just need to enable allow html tables in your settings.

6

I'll second that! Graphs, in particular, are important to surface to people, since it's not obvious how to do them well in Rust.

Each crate we look at has a crate lead, who organizes that crate's discussion thread. They're responsible for hooking people up with other parts of the work to do for the crate. So if you want to dive in on a crate, just volunteer on its thread!

For choosing the crate lead, we'll use this thread (which we'll also use to choose the next crates to tackle). That role is a big commitment, and includes presenting the evaluation at a libs team meeting. If anyone is interested in doing that, just say so in this thread (and perhaps mention which crate(s) you'd be most interested in tackling).

Absolutely! This is very much part of the plan; we want to use this effort as an opportunity to onboard folks at all levels.

7

CommonMark doesn’t have tables, does it?

8

Not natively, but GitHub Flavored Markdown introduced an extension for it. To implement CommonMark we’ll be using markdown-it, which already supports the GFM table extension.

1 Like
10

The tracking issue for memmap is up. Thanks @sfackler and @danburkert.

11

@crlf0710 Thank you for the excellent suggestions. Offhand I agree about all of them and think you can go ahead and add them to the op, maybe with the exception of gc - I don't know much about this crate and the Rust GC story is really immature yet.

Most of the issues filed so far are tagged 'easy' or 'help wanted', and posted to the twir thread. I don't have contributor access to the memmap crate, so none of those are tagged, but there are relatively few filed against it.

12

I’ve updated the schedule through 7/11. I moved error-chain from 6/13 to 6/27 and set @aturon as the lead. @Yamakaky I hope you will be interested in helping with the evaluation and attending the libs team meeting.

To fill out the rest of the schedule I just moved crates from the “on-deck” section, which is now empty, into the schedule. So walkdir is 6/13, mio is 7/11, and gcc is 7/25. All of them still need leads. Please say so if you would like to volunteer.

walkdir is interesting because it could rightfully be in std, probably has some cross-platform considerations. mio is big so the evaluation we should expect to be quite involved. gcc is going to be the first tooling glue crate we’ve looked at.

Feel free to suggest or make tweaks.

@library_subteam let me know if this doesn’t work for you. @carllerche I believe you’ve previously expressed assent to evaluating mio here, and I hope this schedule will work for you.

There have been a bunch of good suggestions for other crates made on thread, and the op should be updated to include them somewhere. I may get back to that later today.

13

I've updated the schedule through 7/11. I moved error-chain from 6/13 to 6/27 and set @aturon as the lead. @Yamakaky I hope you will be interested in helping with the evaluation and attending the libs team meeting.

Of course! Don't hesitate @aturon.

1 Like
14

I signed up to eval the gcc crate.

2 Likes
15

Can I chime in just to suggest a crate? I think chrono is in need of work, and it seems to be the go-to solution for datetimes for many commonly used crates. Especially now that the time crate is deprecated, (btw. chrono still depends on it which I find unfortunate!) I think that much more work on chrono is needed.

6 Likes
16

Yes, I agree chrono is very important, and needs work. I expect to make a call for a more thorough design review of it soon. I think rand is in a similar category.

Like a chrono design review, there are several other useful library tasks that don't quite fit into the structure of the libz blitz that I'm hoping to get moving soon.

17

@fitzgen suggested today that cfg-if is worth a look. It fits into the category of things that could reasonably belong in std.

18

Might I suggest openssl?

19

Ok, here are all the additional crates suggested on thread so far that are not on the crate list.

I am in agreement about most of these, but I don’t know much about daggy, linked-hash-map, or matrix. I think the gc crate is quite experimental and would consider it out of scope today.

20

Can I suggest itertools and nom? Nom might be outside of the lib-blitz scope as it is currently ~2.2 but it could use some love.

1 Like