@jntrnr Thanks for that perspective. It’s really important. Increasing Rust adoption is our most critical goal, and we rarely drill down into our motivations far enough to tie individual decisions to that ultimate concern. None of our effort matters if Rust doesn’t continue to gain new and happy users. Hopefully the more structured roadmapping process we’re heading toward will help clarify that.
1 Like
Ah yes! This makes what you're saying make a lot more sense to me. Periodically, some people try to argue that Mozilla has some sort of nefarious goals with Rust; someone on HN thought that this was some sort of monetization strategy, for example. So I was mostly responding to the Mozilla part, not the rest of it. s/Mozilla/Rust teams/ clarifies a lot, thank you.
I think these are excellent goals.
I'd be curious to see the community response to everything in the proposal except the standardized crates. I don't see anyone against distributing clippy or rustfmt, for example.
Are these goals still served if the community organically evolves a series of metapackages for different domains, as opposed to a monolithic official metapackage? Is it fundamentally different to write a guide about using the async metapackage than writing a guide about using rust-platform/mio? Is there perhaps benefit to having a web metapackage maintained separately from the embedded metapackage? If metapackages would be available to all Cargo users, what value is added by an official Rust Platform metapackage?
Also, I don't maintain any popular crates so perhaps someone could do me the benefit of explaining:
- Why are community crates holding back from 1.0?
- What part of having an official distribution of community crates would resolve the perceived blocks to release a crate's 1.0?
- Are there decentralized alternatives to provide incentives to release a 1.0?
3 Likes
I can speak for hyper: HTTP is a complex problem, and iterating on the design means breaking changes. Especially as the community tries to figure out non-blocking IO. I'm also usually working alone, with occasional contributions (much appreciated!), and in my free time. This makes iterating take longer. A "platform" wouldn't make me tag 1.0. It still needs plenty of work.
6 Likes
I think it would be extremely beneficial to rehtink how one would find a crate to fulfill a certain need. I don’t think bundling a limited set of crates as proposed is a good idea, though. We just need to make it much easier to find crates and I think this could generally be done by just improving crates.io.
Here are some general thoughts about improving crates.io:
- Keyword search on crates.io is pretty terrible. A main cause of this, I think, is that humans are pretty bad at determining what is useful to know about a crate, which results in bad descriptions and keywords. Often, googling just crates.io gives much better results than the crates.io search.
- The search results don’t let you do any meaningful sorting or filtering.
- There is not a good measure of ‘popularity’ for a crate. There’s just the number of downloads (which you can’t sort by), which is a poor metric, because being a dependency of a popular crate just pushes up your numbers. I think a graph-based ranking system could work.
- There is no structure to the crates listed on crates.io. Meaning: no category system, no directory. Perhaps this is a result of a line of thought about everyone using search for everything these days, but that only works if the search is as good as Google. Also, I think developers are more likely to want to browse a structured representation of data than average users.
- Related to points 1, 3 and 4: give users of crates.io more influence in how data is presented. Because crate owners don’t necessarily know how there crate will be used, users need to be able to add useful information to the database. Perhaps users could vote or you can import data from GitHub (stars and forks). Users could add missing keywords or otherwise improve how a crate is presented. This needs to have low overhead (lower than filing a PR at the crate’s source)
7 Likes
I would like to mention that finding an interesting crate is only half the work.
Writing lots and lots of inefficient and error-prone scaffolding boilerplate so that the dependencies you ended up selecting can talk to each other is the second half (because they use incompatible types or incompatible versions of the same types).
A meta-package prepared for a given domain would allow solving both problems in one fell swoop; that’s rather neat.
@dikaiosune - yeah, I like the idea of metapackage “distros”. People that care about certain areas could help curate and ensure the crates work well together.
@seanmonstar - it makes me sad you’re the only person really working on hyper. Things like “http” should be what we as a community rally around solving. They’re core pieces of what it means to program, well, any language these days really.
I already said it on reddit: everybody's working on the compiler because it's somehow prestigious, and nobody is working on the libraries. Piston is one person. Gfx-rs is half a person. Glutin and glium are not maintained because their author doesn't have time. The situation is disastrous.
3 Likes
As for glutin, the reason why it's not 1.0 is that I have actually no idea what I'm doing when working on it. I started the library so that there's at least something. At the time (something like early 2014 I think) the only existing alternative was glfw-rs, which was very tedious to use. Again, the lack of manpower.
Glium depends on glutin, so it can't be 1.0 either. All my other libraries are largely unfinished.
1 Like
I think it's definitely worth collecting input from other community crate authors, but it's starting to sound to me like a significant reason holding back community 1.0's is that there's not enough effort directed at contributions to them, and that they haven't done enough iteration on their API. Is there something (either alongside or in place of the platform proposal) that the Rust teams could be doing to funnel more contributions and dev hours towards these important community crates?
The way that I see it is that we need to grow in general. It's not like we have a ton of people with spare time that don't know where to put it; everyone I see in Rust world is working extremely hard at shipping all kinds of awesome stuff. The best way to lighten the load is to get even more Rustaceans on board, which is why the focus on new users and growth is important.
4 Likes
Can we provide a “batteries included” experience for Rust that doesn’t lead to stagnation, one that instead works directly with and through the ecosystem?
We could start by automatically hosting documentation for crates on crates.io, and if there ever is a set of “endorsed” crates, host their documentation directly on doc.rust-lang.org.
Speaking from Haskell experience, Hoogle is amazing. Part of what makes it so great is that it doesn’t only search through the prelude, but also through core third-party packages. All API reference searches can be served from a single endorsed place, and you can do a cross-package search from that same place. And Hackage serves API documentation for the entire ecosystem. Currently most Rust crates have online API documentation available, but cross-referencing across crates is limited, and there is no uniform search. Having all documentation in one place like Hackage with Hoogle is a game changer.
The biggest open question about what I described above is: how does curation work?
A small feature suggestion for crates.io that I think could help a lot: allow users to “vote for” or “star” a crate, similar to voting in the AUR, starring a repository on GitHub, or starring/+1’ing an issue in a bug tracker. The current metrics to judge packages by, downloads and dependees, are not necessarily good proxies for recommending crates. Downloads in particular is heavily influenced by the number of dependent (popular) crates and by the time the crate has been around. (Though that would be true for votes as well.)
2 Likes
There is no incentive as a library author to publish a 1.0 crate. When you publish 1.0, people expect you to start mangling your code to maintain backward compatibility. Yet, there's no benefit to the library author for doing that extra work. Thus, publishing a 1.0 crate is a non-starter for the library authors.
There's more to it than just the version numbers, though they're important. Rust also needs a shared vocabulary. We're seeing issues where there are lots of ways to do something. While this is great in a way because there's a lot of choice, it's also is a drag on developer time trying to find the right one.
Could you point to some study, even an informal one, that shows this is true? In my experience, it isn't a significant timesink.
For example, by having an established set of traits and capabilities we build networking on, we can build even more sophisticated networking functionality on top it because we know it's stable and supported. Of course, replace "networking" with a variety of common programming tasks.
Conversely, I'm looking forward to people building new replacements (perhaps the same crates refactored, perhaps new crates) for almost all the currently-commonly-used networking libraries. The current generation is useful, but I think there are significant improvements that can be made.
With that shared vocabulary, we can do things like write guides and tutorials that assume these features are available. This lets us make much better starting examples help users quickly solve real problems. This saves them time and gains the community another Rust developer in the process.
We don't need the Rust teams to spend time on that. The early adopters of the various libraries do this and will do this. This is relatively easy to do and is a good way to start contributing to the Rust community. Conversely, it's much harder for a newcomer to add alignas to Rust or add 64bit * 64bit -> 128bit multiplication intrinsics to Rust or do other things that the Rust teams are really better suited at doing.
give crates a reason to hit 1.0
To the extent that this gives crates an incentive to publish 1.0, it seems coercive (no doubt unintentionally) and not in the best interests of the library maker. It would be better to find incentives that actually benefit the library makers (e.g. money).
I think in many cases this is not actually an issue with maturity. In my opinion, many authors are simply too conservative regarding versioning. At the time of writing, none of the most-downloaded crates on crates.io are stable, yet they are the most widely used and battle-tested crates out there. 1.0 doesn’t have to be perfect. There’s no shame in releasing 2.0. Or 52.0 for that matter.
Make Cargo colour version numbers < 1.0 red and version numbers >= 1.0 green. Although that might actually scare people away from unstable packages instead of nagging authors. Perhaps only colour the version number for the root crate?
1 Like
This project is already largely underway, see
2 Likes
Thanks, everybody, for the excellent feedback so far – both here and on various other forums. I’ve been reading and digesting all the comments.
To state the obvious, it’s quite clear that the overall response to the proposal as written is negative. Which is fine! I think people are raising a lot of good points. On the other hand, there is definitely room for improvement in the areas under discussion (e.g. discoverability, maturity, interoperability), and many people have been proposing lighter-weight approaches for doing so.
Given the large number of comments here and elsewhere, and the repeated themes, I’m not going to try to respond individually. Instead, I’m going to put together a short follow-up post, gathering the downsides people have pointed out, looking more closely at the goals, and summarizing some of the alternatives being proposed.
9 Likes
The way it is described above seems like this would attract organizations that don't want to contribute significant effort towards maintaining the libraries they use--i.e. freeloaders. But, that's not the kind of organization that we really need right now, in the world of Rust library development. We need organizations that are willing to get their hands dirty and improve stuff with the people who are building the libraries, or who are willing to at least sponsor the maintenance of the libraries in a meaningful way.
I was just talking with a friend that does a lot of node.js stuff, and he said almost all the negativity in his project comes from people who are demanding fixes be backported and/or new NPM releases of his library be made immediately, and who never contribute positively. I've seen how that works myself firsthand. That's not a position I want to encourage Rust library authors to get into.
2 Likes
This is exactly what I was going to type up but even more eloquently said. I would add that if the Rust language committee or whoever would be in charge of such a list sees a gap in tooling or libraries or what have you, then there should be empowerment to fill it in an appropriate way until something else supersedes it. By simply having a well curated list that gets a bit of official support like documentation and the like, no one who is just upgrading a version gets put out by something no longer working as expected (say behind a corporate firewall). They will have to figure out the normal path to get their libraries and when the community starts preferring a different library, they still get the old library they were pleased as plums with.
If the issue is how to make it easier to get libraries for people behind firewalls (or some otherwise restricted environment), then address that issue explicitly.
There is no incentive as a library author to publish a 1.0 crate. When you publish 1.0, people expect you to start mangling your code to maintain backward compatibility.
While we should definitely also talk about ways to encourage crates to stabilize, I'd like to at least make sure we fully understand the end-user experience before switching topics. Both are important and need to be understood.
You also seem to have a very strong take against crates.io as a concept, at least insofar as it becoming a place that people can find stable, supported packages. I think that's a concern we're trying to address, and I'd love to hear your thoughts in terms of how best to address it.
Could you point to some study, even an informal one, that shows this is true? In my experience, it isn't a significant timesink.
While I'm not sure I understand what you're asking about specifically here, my comment is more about the basic mechanics. It's building out of composable parts. If the crates don't work together well, you spend time stitching them together. Having the shared vocabulary also means we spend more time building the new thing rather than recreating the old thing.
The survey results underscore this. The need to cover major programming areas like UIs, networking, async I/O, etc are the specific areas that need strong support.
To the extent that this gives crates an incentive to publish 1.0, it seems coercive (no doubt unintentionally) and not in the best interests of the library maker.
Are you saying here that you're worried that library authors will rush to hit 1.0 before their ready? That's definitely a valid concern, though I hope some of the curation ideas mentioned here help ameliorate it.
1 Like
Publishing a 1.0 creates more work for the library author in terms of support costs and maintaining backward compatibility and whatnot. (Otherwise, the version numbers are meaningless.) Including a library in a curated set on the condition that they publish a 1.0 means they have to choose between being doing that additional work or being an outcast. Both are bad choices for people that just want to keep making the code better. It would be great to find other incentives.