Yeah. I am definitely not in favor of polluting namespaces randomly.
If we want to have the platform in Cargo, it should pub extern crate the libraries.
3 Likes
I want a curated platform because I want to be able to find a library without risking depending on some fly-by-night left-pad with terrible security issues that will stop being developed 6 months for now.
My day-job employer, and other large corporations, are even more conservative about these things than me. They will not introduce a new library without an extensive process (which does not actually check library quality in any way, except by reducing the number of libraries used). A Rust platform would provide us the guarantee of support we need.
2 Likes
It might be better to organize this project around the best way to distribute official rust-lang crates, a system which already exists, rather than introducing a broader notion of ācuratedā crates.
Have you considered how much better your contributions to this thread would be taken if you had just posted everything you wrote after this, and not this part at all? I participate in the Rust community in my free time, as I believe you do as well, and seeing comments like these makes me unhappy, even if they're directed at someone else. Its sarcastic, ungenerous, and most of all, unkind.
Maybe this is just more 'politics' to you, but these comments have drug down the tenor of this entire conversation.
4 Likes
Itās great that the version stagnation problem is being acknowledged. Itās not entirely clear to me that itās entirely solved, though; libraries will start using new features (in the language or in libstd) that make it hard to update the library but not the compiler, and the compiler occasionally becomes more strict when discovering safety holes, possibly making it impossible to update the compiler but not the library.
(I donāt have a solution for it either, but I feel the problem should at least be considered when evaluating proposals in this sphere.)
I would also prefer a curated, consensus based website listing bundles of great Rust packages (and maybe their documentation) over a Rust Platform Package.
The only place where I could see a great advantage in metapackages would be the case of ālibrary ecosystemsā like Piston or Hyper/Iron. But even then, if Iām doing a 2D single player game Iāll need a different set of Piston libraries than if Iām doing a 3D network game.
Having a good website listing great combinations of packages, with a simple ācopy this to your Cargo.tomlā instruction, would suffice in my opinion.
4 Likes
Good feedback, and I agree. I didnāt mean anything more negative than āIsnāt it obvious?ā Actually, if I would have typed that first I would have seen how condescending it is on my own. Sorry.
5 Likes
I am not sure about metapackages.
I think something like Yeoman where you just get a bigger, batteries included, maybe even with some demonstration code, starting base for your project might be more useful and a more decentralized and agile approach instead of having one definite ārust-platformā.
Think of some kind of project template generator, where you first choose one specific generator for a specifc kind of project, like āgame development starter kitā and then you are asked some questions by the generator about further customization of the template like ā3Dā or ā2Dā. After this, the generator compiles a nice starter project with good libraries that work well together in proven versions. The effort to test these generators lies by the developers of those. Via a ācrates.ioā like web platform you could rate, flag and comment on them.
I think having metapackages can be useful, but having some sort of official Rust metapackage is not a good idea. Iād rather that endorsing certain crates and curating various lists and metapackages be left up to third parties. I especially do not like the Rust platform being automatically injected into new projects or being installed automatically with Rust. Sure there should be more effort to point new users in the direction of places where they can look for high quality batteries, but donāt force batteries on the user.
Where is the sense that there is this strong desire for a batteries included stdlib coming from? It may have been reflected in the recent Rust survey, but I donāt recall. This whole proposal seems like a lot of work and complexity just to provide a batteries included experience. Personally, I like stdlibās very slim approach, and would not want any sort of official batteries included distribution. My general experience with the Rust community has been that they agree with that. I understand that the intent of this proposal is to make the language easier to learn and more productive for new users, but Iām not sure this is a good way of achieving that. Metapackages donāt seem necessary to me, and the idea of community crates being chosen to be in a āplatformā distribution will introduce unnecessary politics into the community.
The root problem this tries to address, as I understand it, is to make it easier for new users to answer the question, āWhat library do I use to do X?ā I think a better approach to this is just to have a website with carefully considered UX that answer these questions. Likely, that website could even be crates.io. Trying to make the answer to this problem a technical solution involving tooling doesnāt really make sense. Even the suggested benefit of being able to test a group of very commonly used libraries together seems unnecessary. If the libraries are really that commonly used together, bugs/stability issues will get quashed anyway because they will be more readily discovered and fixed. In other words, this proposal seems to make unnecessary technical changes for what should happen organically.
On a smaller point, Iām not a fan of the phrase āRust platform.ā The word platform is become increasingly diluted and meaningless in the computer technology world, and to me it evokes a cynical sense of annoyance with all the tech startups touting themselves as āa platformā when often itās just āa serviceā or even āa website.ā I understand that āplatformā is meant in the roughly the same spirit as the Haskell platform, but given the mixed feelings that community has about it, that doesnāt seem like good lineage either.
8 Likes
Having used Java and having experienced how you learn to replace the JDK URL parser with something else (first Jena IRI and now Galimatias), the JDK HTTP client with something else (Apache HttpClient, potentially another newer lib whose name escapes me), the JDK encoding and i18n libraries with something else (ICU4J), etc., Iām worried about entrenching first-published libraries beyond their natural first-mover advantage and making it even harder for better later-comers to be adopted.
OTOH, compared to e.g. C++, where even strings arenāt standard across multiple real large code bases (e.g. Gecko and Qt) having standard types especially for foundational stuff can be very useful if the foundation is right or right enough. Java having a single (though not optimal) foundational notion for Unicode (UTF-16 even though UTF-8 would be optimal) and a common efficient and spec-wise correct (though not design-wise optimal) XML API (SAX) was a key reason to develop Validator.nu in Java as opposed to e.g. Python (whose notion of Unicode varied between UTF-16 and UTF-32 based on how the interpreter was compiled!).
Still, at this early stage, Iād point to crates.io plus some kind of āwhatās good on crates.ioā documentation as the ābatteriesā instead of creating a layer of official mechanism on top.
As @matklad points out, the visibility / community-standard API issue and the code distribution issue are distinct. Cargo is already awesome at the latter. I think it would be a net-negative if the Rust team were to ship pre-built crates instead of having the users of the crates build them with appropriate settings (esp. panic aborts).
As for visibility / community-standard API issue, I share @briansmithās concerns, but Iām biased, because lately, Iāve been writing a second-to-market crate for a topic for which a crate already exists. But to talk about his code rather than mine, I think itās problematic that ring pulls in the legacy SSLeay license dependency, but Iād much rather see that issue solved (some sort of audit to conclude that all affected legacy code has been removed) than to have something technically less good entrenched in the mean time.
At this time, Iād prefer the visibility / community-standard API issue be addressed by the docs linking to Awesome Rust.
5 Likes
I am fairly new to the Rust community, so take everything I say with an extra-big grain of salt.
I come to Rust as a systems programmer who works primarily on C (and some C++). One of the things that is appealing to me about Rust is the minimal nature of the language. The std library is slim, and anything else you import, you import selectively and consciously.
The whole metapackage system seems like it will hurt this, since the default mentality will become to depend on whole metapackages instead of frugally choosing dependencies. This will lead to more dependency bloat. Even currently I think more rust libs should actually try to be independent of other rust libs, instead of relying on them just for minor convenience.
A better system would be a more community-driven crates.io, for better discoverability, user reviews and organic curation.
6 Likes
I have used āpackagesā for quite a while now in C++, as the company I worked for had developed its own system, and it worked with a slight difference compared to this proposal.
Instead of having āmagical dependenciesā, I would propose a new section package:
[packages]
"rust-platform" = "2.3"
[dependencies]
"regex" = "packaged"
"other" = "1.3"
This is slightly different:
- it makes it clear what is a package and what is a regular crate
- it makes it explicit what are the dependencies (completely obviating the massive download/compilation issue), and which dependency is versioned by a package or not
Thereās also something subtle going on here: it does not let you specify which package the library comes from. I am unclear whether this is good or it would be better to use the package name instead (allowing cherry-picking). Itās not clear how beneficial drawing from multiple packages would be.
It might also be interesting to have a package of āpackagesā: specifying multiple packages, cherry-picking the needed crates, overriding a few, etcā¦ in order to deliver a curated package for a specific purpose.
Regardless of the rust-platform idea, the idea of packages is very helpful for large organizations. It makes it easier for an organization to cherry-pick versions of crates that work well together and deliver a package of their own for their own developersā consumption, obviating the need for each team/internal project to repeat this curation/compatibility check work.
3 Likes
Could you elaborate a bit more on how this avoids the problems you cited with a platform? Specifically, how does this avoid introducing unnecessary politics into the community in a way that you think would happen with a platform?
As with several other people here, I've had some experience with the Haskell Platform, and have some reservations about a set of "blessed" packages becoming de-facto standard packages, and think that a certain amount of that is reflected in the current state of the Haskell Platform -
Note: as of 8.0.1 there are two download options available ā minimal and full. The minimal option is currently the generally recommended one. It does not include any additional global libraries beyond those packaged with ghc, though it does include all tools. This ensures maximal compatibility with a variety of library sets. The full option is useful for those who prefer the "classic" platform behavior with a broader set of preinstalled libraries, and especially serves those well who want full-featured installers in situations where network connectivity should not be taken for granted.
Having said that, I like the idea of meta-packages - I could see myself constructing a personal meta-package over time, with experience of crates that I'd used myself, almost as a reminder of what crate's I've had good results from. And a significant part of the Haskell Platform's problems came with conflicting versions... Would meta-packages constrain the versions of crates that were used, or would that be left up to Cargo? As you said - the devil's in the detail...
I donāt believe a metapackage is necessary. However I like something like awesome-rust, but up-to-date and maybe package rankings driven by github stars? Iām fairly new to Rust and when Iām deciding on a package to use, if there is none obvious candidate, I look upon the usage examples and documentation of each package and consider stars ammount and the stability of commits since beginning. Also the stability or disclaimers in README that say it is not stable.
I suggest some site, that would contain these criteria and that would be updated automatically from cargo and github. The information is already provided there, but could be upvoted and downvoted by the community, much like reddit posts. This would be represented by some score ?
Also, I disagree with something like C++ boost. It is a giant library, much like jQuery, where jokes are being made of plugins for adding integers.
In general Iām a fan of anything that makes it easier to adopt Rust (whether in free time or at work, but especially the latter). And Iām probably more in favor of this proposal than many others in this thread.
However, perhaps there are some ways to improve some of the issues presented without adding as much new work to the subteamās plates?
For example, I think that crates.io could have significantly better discoverability by leveraging the metadata that crates provide. Being able to browse by keyword, explore popular keywords, etc., could not only make it easier for the specific high-traffic cases presented in this proposal, but also improve discoverability across the ecosystem. This might be significantly cheaper in dev hours than maintaining integration tests and backporting security patches and bugfixes for a growing set of community crates. (which gets at another issue ā who does the work in the Rust Platform to fix, for example, a TLS issue thatās been fixed since the old version of hyper in the platform?)
In terms of selecting versions of libraries with compatible dependency versions, is this something which requires a new heavyweight process to improve? I would imagine that effort spent pinning versions together could also be spent keeping popular librariesā dependencies in sync for their current versions. Is there a specific reason why the āofficialā choices need to be held back, as opposed to contributing to keeping the popular libraries up to date? IMO, it could be more in line with the Rust culture to as a community keep the most up-to-date versions of libraries in sync with each other.
I also think several of the parts of this proposal are fairly uncontroversial, and I think it may be beneficial to consider some of these things as separate initiatives. Itād be much easier to discuss alternatives for each individual part of this.
I do really like allowing rustup to distribute rustfmt, clippy, language bridges, etc., with better support for cross compilation and other tooling. I wonder to what extent that may be orthogonal to the idea of a āblessedā set of crates. Do these things really need to be considered together?
Does having cargo support for metapackages mean that there necessarily needs to be an official metapackage? I see value in supporting this feature regardless of the direction this proposal goes.
3 Likes
I fully agree.
Moreover, I would rather the community experiments and polishes meta-packages, and how to use them, before it gets used for anything semi-official.
1 Like
Before we really dig into the solution, I wanted to at least enumerate the problems.
Our primary motivation is to make Rust as successful as possible. I donāt think we say that enough. I donāt mean in a āRewrite it in Rustā way. I mean in a āpeople should always be able to choose Rust for their projectā. We should be removing any blocks in the way of someone picking up Rust.
Weāve already been doing great work here with things like rustup, clippy, and all the IDE plugins. Weāre hacking away at where we see these blocks, but here I wanted to spell out the problems that motivating things about the platform.
From the survey results, we got feedback on what things need to be addressed:
- Learning curve
- Lack of IDE support
- Immaturity of libraries
Learning curve was a huge point of feedback. Looking at the comments, people mentioned it more than anything else. Why is learning curve important?
While some of us have time to really dig into Rust, most of us simply just donāt. Those of us who have put the time in know that Rust is worth it. You get out what you put in, and sometimes the result can be surprisingly efficient even in your first attempt. This makes Rust fairly unique among languages. The problem is that learning curve sharp. Some users describe it as the ānightmare weekā before they understand enough of the language to be effective.
The lack of IDE support is important, and Iād happily talk about that for hours, but I wanted to skip to the immaturity of the libraries and dig into that.
If you look at crates.io lots (most?) of the crates there havenāt hit 1.0 yet. This is a signal to many developers that Rust simply isnāt ready for prime-time. These programmers donāt want to spend time messing with APIs that are breaking underneath them. Theyāre too busy trying to solve their actual problem instead of dealing with maintenance issues. Once they see this, Rust is a non-starter.
Thereās more to it than just the version numbers, though theyāre important. Rust also needs a shared vocabulary. Weāre seeing issues where there are lots of ways to do something. While this is great in a way because thereās a lot of choice, itās also is a drag on developer time trying to find the right one. Additionally, itās simply easier to ābuild higher from the 17th floor than the ground floorā. For example, by having an established set of traits and capabilities we build networking on, we can build even more sophisticated networking functionality on top it because we know itās stable and supported. Of course, replace ānetworkingā with a variety of common programming tasks.
That stability opens up the doors to a lot of new users. Going back to the comment about people not having enough time. With that shared vocabulary, we can do things like write guides and tutorials that assume these features are available. This lets us make much better starting examples help users quickly solve real problems. This saves them time and gains the community another Rust developer in the process.
If we can solve the learning curve, give crates a reason to hit 1.0, and build a shared vocabulary, weāve given Rust a much stronger platform to build on and more quickly get new Rust users going.
9 Likes
@aturon, I think a lot of the reservations in this thread wrt stagnation and blessing could be addressed by providing a preliminary idea of which libraries you would nominate for the platform, to give us some idea of its expected scope. In particular, I sort of get the impression that you donāt intend this platform to be anywhere near the size of the Haskell platform (which is 30+ packages).