Calculating which 3rd party crates are good candidates for "std" inclusion via "left-pad index"

It seems to me that some of the empirical evidence and especially the methodology (esp. using downloads is a questionable metric) behind it is under question. Many of the crates are also quite large ones.

Are we ignoring the signal? In my view things like Tracking issue for eRFC 2497, "if- and while-let-chains, take 2" · Issue #53667 · rust-lang/rust · GitHub are in fact responding to the signal by extending control flow in the language to support a common use case.

Rust is opinionated about lots of things. For example, we do not have exceptions (we have some sugar, those are great, but those are not the same language feature as in e.g. Java). We don't have global type inference (a great feature if you ask me). I don't think it's very interesting whether we should be opinionated or not. I understand that you like this feature and we can discuss that, no problem.

But I do and have used it (I said so before...)? The crate exists and it seems to work, so why does it need to be elevated into the language (especially when it has maintainability drawbacks)?

4 Likes

As an alternative to adding stuff to the stdlib, maybe the Rust organization could respond to the "left-pad index", and other signals that crates are particularly important to the community of language users, by extending organizational support to the maintainers of those crates. That could take many different forms, but usually what maintainers need most is funding and helping hands.

2 Likes

I don't see that microcrates would be a bad pattern in themselves. They are easy to ridicule (left-pad style), but that's superficial. In the spirit of controlling compilation times, I'm one of those looking for crates that do just what I need and not more (which does not favour conglomerated crates).

Everyone can see the high rdep count on lazy_static, that's a vote for something that should probably be in std (in some shape). I don't think the lower rdep-scoring crates are sending a that strong signal, even matches has "only" 116 rdeps.

3 Likes

With all due respect, you seem to be proposing to ignore the signal in the case of lazy_static.

Because it's an extra step that everyone using the feature has to go through, it increases the compile time of crates, and it increases the surface area of actors that you have to trust. I also don't agree that lazy_static has maintainability drawbacks, as a feature. What does have maintainability drawbacks is forcing everyone to go through a microcrate just for that feature.

I do kinda think something not entirely unlike lazy_static should be in std, but not just because the crate is heavily used. Rather, my concern is that it's commonly needed and the stdlib does have something for it (std::sync::Once) but you need to write unsafe blocks to use it. I actually wrote a proposal quite some time ago.

(This isn't to say that the stdlib has to provide safe wrappers for every unsafe primitive it offers; but when the stdlib offers an unsafe primitive, and there could be a straightforward safe wrapper for at least 80% of what people use that unsafe primitive for, I think there's a strong case that the stdlib should offer that wrapper.)

4 Likes

what exactly is holding us back to try something like stdx but with a different goal. Other than having something like a complete "batteries included" replacement – which is very ambitious – we could vendor said libraries in a "microcrate" that is (un)officially blessed and see where things are going. If the adoption rate is very poor, adopting those crates into std was a misleading idea in the first place. If it wasn't we could reiterate about the idea to adopt that "microcrate" into std at a later point but this time we could make a more informed decision. We could learn if people tend to switch to newer versions instead of the freezed ones in "microcrate" or switch to better ones altogether. I think a "microcrate" that is just combining said crates is not a bad idea and could replicate the experience and adoption if we would pack those into std without committing to have those forever inside std which, i think, is one of the main counterpoints – which i agree with.

Maybe not lazy_static though, it was good but we have once_cell now. It would be great to have something like the Lazy type in std.

2 Likes

Incidentally, I've been thinking about writing an RFC for std::[cell|sync]::[OnceCell|Lazy] this week, so here we go (early unfinished WIP).

TL;RD:

  • yes, we need to add lazy initialization to std, as it is important enough even in the world with perfect const (which we all want) and magically lazily/before-main -initialized statics a-la C++ (which I personally don't want).
  • lazy_static is suboptimal API: we can do better now, we don't need macros.
  • API of the once_cell crate seems canonical (i.e, you can't really do anything smaller/simpler, modulo naming).

EDIT: RFC PR

20 Likes

I think another aspect that's missing is whether these crates are the dominant and polished solutions to their respective problems, so that when std picks one "winner", most users will be happy with it.

For example, cfg-if, atty, and num_cpus seem to have one uncontroversial solution, and if they were in std, I don't think anyone would mind.

But OTOH logging is a big enough problem that it can be solved in multiple ways, and there are multiple crates competing. log works for Patrick, but I write servers, and it causes headaches for me, so that crate is not one-size-fits-all.

3 Likes

If std incorporates one particular solution, that still doesn't mean there's no room for other solutions. It's still okay for a minority of users to come up with, maintain and use something else, as long as we can make an easier/safer choice for everyone else.

Requiring that any std inclusions are perfect seems like a non-starter to me.

That's fair! the point I'm trying to get at—which I didn't articulate properly—is that I can't think of a logging interface that can be introduced into std without locking-in a specific instrumentation model.

1 Like

Perfect? Of course not. Rust itself is not perfect.

What I don't want is another situation like std::mpsc::channel or Python's urllib3, where there's an uncontroversial "better" option available outside libstd. Replacing the standard library toolkit with a third-party option should be reserved for niche use cases, not considered a universal best practice.

But I'm not arguing against including log in the standard library, actually. It already is a de-facto standard, to the point that its "competitors", like slog and tokio-trace, provide compatibility layers for it. The log crate seems like one of the best ever candidates for libstd inclusion, because of its nature as "glue" between libraries and their host applications.

6 Likes

Generally on board with @Centril regarding lazy_static!, singletons are like cancer, once you have them they spread and corrupt everything they touch. Notice what motivates your example where singletons are a good idea, someone designing OpenGL used singletons and now you have to use them too. Similar story for example with a lot of C++ libraries that interact with OpenSSL, they too have to make heavy use of singletons because OpenSSL choose to do so. At the same time, sitting on an island of 'Rust is beautiful and great' while ignoring existing software and its interop eg. OpenGL is not a particularly useful approach.

With all that said, communicating intent and best practices with language features, or their lack of, is a tricky subject and in my experience the biggest influence comes from educational material, especially things like stackoverflow answers.

2 Likes

It would be better to just have documentation search across all crates, with results grouped by crate. That way you can easily identify the crates that have the functionality you need. Better than bloating std, anyway.

4 Likes

Something I feel got missed in this discussion: it's a lot more than just arrayvec and nodrop that have been supplanted by language and/or standard library proposals. Just to give a few examples:

5 Likes

Folks have already mentioned transitive dependencies, but another source of this is being included as a dependency for a crate that's not hosted on crates.io. I ran into this with a crate of mine being included in cargo which made its download numbers go way up.

To the original point--I don't think that "lots of people use lazy_static" means that we ought to import it directly into std, but it does mean that lazy_static is clearly filling a need for a lot of Rust users and having some sort of equivalent functionality in std would be useful. I bet we'd get a lot of mileage out of taking the crates on this list and having some Rust experts sit down with the authors to figure out the salient bits each crate implements and whether they'd be a good fit in std, or if it would fit better with some small changes.

4 Likes

We've already done the experiment to answer the question "do you really need singletons in Rust?" The answer: yes, you absolutely do. Everyone just uses lazy_static or once_cell, despite the friction involved in doing so.

We have failed at the goal people apparently think we have of encouraging people to not use singletons (which is a goal I disagree with, but anyway). I don't think pretending we'll succeed at that goal if we just keep the status quo is productive. Rust should pick battles it can actually win.

12 Likes

These forms of singletons do avoid life before main though, which is a good thing.
(You can still get called earlier as a global allocator though.)

2 Likes

Wasn't there a proposal at some point for assert! to recognize certain top-level operators and show their operands, so that assert!(a == b) could give the values of a and b just as assert_eq!(a, b) does?

2 Likes

That would be RFC 2011: generic assert.

2 Likes