Quality declarations for Rust code?

1

In my opinion, Rust now has a large enough ecosystem that it might be a good idea for crates to declare their quality levels like ROS does. This would augment semver requirements with information that would help end users decide if a crate and its dependencies are mature enough that the crate can be relied upon in their application. Excerpting from there:

There are five quality levels below, each roughly summarized as:

- Quality Level 1:
    - highest quality level packages that are needed for production systems
      e.g. rclcpp, urdf, tf2, etc.
- Quality Level 2:
    - high quality packages that are either:
        - on the way to 'Level 1', or
        - are general solutions used by many people, but are only sometimes  
          used for production systems
    e.g. navigation2, rosbag2, etc.
- Quality Level 3:
    - tooling quality packages
    e.g. ros2cli, rviz, rqt, etc.
- Quality Level 4:
    - demos, tutorials, and experiments
    e.g. research packages, demo_nodes_cpp, 
    examples_rclcpp_minimal_publisher,  etc.
- Quality Level 5:
    - default quality level
    e.g. for code that fails to meet requirements in 'Level 4'

While each quality level will have different requirements, it's always 
possible to overachieve in certain requirements even if other requirements
prevent a  package from moving up to the next quality level.

I think that the ideas that ROS is suggesting could actually be refined further and at least partially automated. I.e., a new key could be added to Cargo.toml (quality) in the package section that is a numeric value in the range [1, 5]. If it's missing, then it defaults to 5. A crate's effective quality level is the maximum of the quality level of it and all of its dependencies, which can be linted at compile time.

Note that quality levels augment semver, but they do not replace it. For example, version 2.3.45 of crate Foo might have a quality level of 1 or 2, but version 3.0.0rc3 might have a quality level of 4 or 5.

I know that this idea is incomplete, which is why I'm not trying to put it on the RFC process. However, if it appears to have enough interest here, and is refined enough to make sense for rust, I can go ahead and write up an RFC for it.

1 Like
2

What exactly is the advantage of formalizing a specific scheme as part of cargo? cargo-crev code reviews also cover "production readiness," so attaching it as cargo metadata does three main things that I see: 1) provides the author's intent 2) prevents encapsulating a lower-confidence library with a higher-confidence library, and 3) makes it completely immutable for a given release version.

The first benefit — surfacing author intent — is useful, but this is also hopefully communicated through the normal overview documentation.

The second point could be a positive or a negative, depending on how you see it. I think it's reasonable for e.g. a Rank 1 to use Rank 2 or unranked components, and take on the burden of making sure the subset they use is Rank 1 quality. You could also say that this prevents code not actively advertised as "for production" from being used in production (but also, it will be anyway... the industry is built on shipping fast and hopefully fixing later).

The third point I think is problematic. The easy example is some library which is published at Rank 2, and then over time is shown to not have any defects preventing a Rank 1 classification. Now, instead of just updating documentation and/or recommendations, etc. to say that the library is battle proven and production quality, a new release has to be cut just to update (hopefully just) a single line of metadata.

Personally, I think ranking like this would be ill served if it were both self regulated and enforced. Such rankings are most useful imho when provided by a singular, consistent party, and must by necessity provide for exceptions, as social constructs don't divide neatly into strictly delineated boxes.

5 Likes
3

The main purpose is to make it more of a mechanical process, but I'm forced to agree with your last point:

What's more, you reminded me of cargo-crev, which I had forgotten about.

OK, given that not nearly enough people are using cargo-crev, is there a way of making it more official? IIRC, several tools have been absorbed into the official tools chains over the years, so I think that's possible, but I don't know if it's preferable.

4

You may be interested in the extensive discussion and evolution of RFC 1824.

1 Like
5

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.