Expansion of standard library

This would REALLY simplify embedded development with Rust, since I could depend on the useful platform-independent modules in std, and choose later to add fs, net, and thread ad-hoc/as needed.

Instead, I can only depend on core. If I want Vec or HashMap, I need to implement all of the above and more for my platform. :face_with_symbols_over_mouth:

I do absolutely want to see better handling of no_std crates in the ecosystem, and better ways to build subsets of std. That’s being worked on today, and thank you for calling attention to it.

9 Likes

I must be missing something important, but isn’t the point of alloc to provide a middle ground for no_std crates that want memory allocation without bringing in the rest of std?

In an ideal world, other pieces of std would be similarly split in independent modules. I think @brson used to talk about similar ideas back in the day.

Yes, alloc provides collections, I misspoke. A specific example of what I’m talking about is std::io , e.g. what core_io tries to address. Which naturally leads into stdout and println!.

1 Like

For comparison I’m working on an async daemon controller. This includes

  • Authentification (TOTP, bcrypt)
  • Local storage (sled)
  • Http-Server with API (actix-web)
  • Async Process Controlling
    • Round-Robin buffer
  • All as actors (actix)

So except for the async process stuff I’d say a fairly normal thing for a webservice.

Currently at 296 Crates, 27 root crates. Thanks to the sccache authors this is quite ok to compile.
Cargo tree output
And I’m still happy that I’ve got a full choice of what I want to use for this, even more as long as a trimmed down std isn’t available. (Or do you want to ship a std-https server along with the one you’re really using, because http 3+ came into existence)

I can run cargo update daily, thanks to the activity of the crate ecosystem :slight_smile:

4 Likes

What exactly do you see as lightweight ?

  • http 1
  • http 1.2
  • http 2
  • (quic)
  • compression
    • brotli
    • flate…
  • TLS
    • rustls
    • libressl
    • openssl
  • cookies
  • multipart
  • (server: websockets)

These are already big features. Thinks I’d still expect from a “simple http server”, because if not it’s not simple for me to use, it doesn’t “just work” out of the box.

Edit: For servers probably Http 1, 1.2 and cookies are enough for 90%, as you normally proxy everything through nginx.

2 Likes

Because I don’t believe the people who want to cram libs into the std have legitimate needs. I would really like to know these legitimate needs. So far what’s been articulated in these threads is “my manager says it makes us safer for some reason”. This is absolute nonsense. We know it’s nonsense. We know there is nothing magical about the std. We know that putting lines of code into the std doesn’t transform it into “high quality” code. How is that metric even measured? There is nothing spooky about libs outside of the std. It’s open source. You can look at the code. Thousands of programmers are using them. With SV and source control you have infinitely more flexibility to address any security or code quality issues that you have when libs are not in an untouchable section of the ecosystem.

That's a bit brash. People who want to avoid non-std code have some legitimate arguments (especially when we're talking about indirect, transitive dependencies):

  • While being in std doesn't make code magically "high quality", I posit that it does increase the number of eyeballs that review that code (and probably increase the number of tests exercising that code compared to your average crate).
  • It reduces the likelihood of a left-pad scenario, where some dependency goes rogue.
  • It reduces the likelihood that a library will go unmaintained. The std is maintained. Security issues are widely broadcast and fixed in a timely manner.
  • The license terms of the std are clear: MIT + Apache. License terms of dependencies can vary (and even change over time).

I generally favor a smaller-ish std, but I also avoid taking on dependencies because of the risks associated with each new dependency. I also audit each dependency I take on so I can get a better assessment of the indirect transitive dependencies I'm taking on (which is where I think the real risk is). It's fine to disagree, but you're being uncharitably dismissive.

6 Likes
  • While being in std doesn’t make code magically “high quality”, I posit that it does increase the number of eyeballs that review that code (and probably increase the number of tests exercising that code compared to your average crate).

All experience has shown that it decreases the number of eyeballs. Creating the adage that the std is where libraries go to die.

  • It reduces the likelihood of a left-pad scenario, where some dependency goes rogue.
  • The license terms of the std are clear: MIT + Apache. License terms of dependencies can vary (and even change over time).

What great legitimate concerns and entirely out of proportion and not warranting freezing libs into a language. Do you depend on a particular lib for your enterprise™ multi million dollar operation? Fork it. Or compile and statically link the version you rely on. You should prefer to do that anyway.

Especially for a language who's ecosystem already provides all your basic needs with no excessive licensing restraints.

  • It reduces the likelihood that a library will go unmaintained. The std is maintained. Security issues are widely broadcast and fixed in a timely manner.

No, all experience has shown the std is what goes unmaintained. The std has to meet stringent compatibility criteria. Security issues and bugs go into 4 year backlogs in the std.

I generally favor a smaller-ish std, but I also avoid taking on dependencies because of the risks associated with each new dependency. I also audit each dependency I take on so I can get a better assessment of the indirect transitive dependencies I’m taking on (which is where I think the real risk is). It’s fine to disagree, but you’re being uncharitably dismissive.

The std is a dependency. The dependencies do not magically disappear because you moved the code to another folder. It doesn't disappear because you gave it a new designation. The std is a social phenomenon. It offers little more technical benefits than big name labels on similarly functional products at a higher price but with a Brand Name logo.

4 Likes

I want to reply to that. For context, I'd be opposed to put lots of things into std, my main gripe with it being that it would put more work onto the desks of people who have enough on their plate and their time's better spent on design work or implementations of things like async/await and such.

That being said, here's my personal situation: I'm not a programmer, but I write some tools I need. Our IT is "less than helpfull"(tm), which overall means that I have to do with the things that are already installed on my workstation. There's basically no way to get anything else, other than fighting a years-long process of approval for every little bit an piece (large company, not dealing in software...). If python did not have such a large stdlib, I'd have been lost several times. E.g., when I needed to parse some xml, the functionality was there. It felt awkward, it was maybe dead or something, not efficient or something... but I did not need to hand-roll something, which I probably could not have done. So I am glad for the python stdlib, even though it mostly serves as an example what rust should not become.

I don't think rust should cater to my usecase here (for several reasons), but dismissing it as "not legitimate" is unwarranted.

5 Likes

I went through the same pain points as you did, in my company. Years lost not in choosing the right dependencies, but in wrestling with the imports from the std. Faulty code, bad api and awful optimization. Ultimately, we had a fallacy in the thought process behind the people coming up with the organizational dictates. Years to realize the merits of favoring the std was based on social bias rather than any technical or organizational objectivity. We finally won out the argument simply through sheer pain and secured a company depository for third party dependencies that we pull from. What we used from the std was always inferior in some way. People were always coming up with new and better and faster implementations while the std lagged behind.

What I find frustrating is the people who periodically voice demand for their usecase getting stamped with std™, is that they assert it makes them objectively safer or using objectively better code. Then of course if the Rust community is not onboard, they threaten to “leave” to their previous language of choice. This is an easy demand to dismiss and it gets exceedingly easier to dismiss every time someone says they will storm off to golang (which has a sparse std anyway) if rust doesn’t do it. Meanwhile, there is already a crate to do exactly what they want, faster and better maintained. It’s baffling.

9 Likes

That's not the case for Rust's std, at least for security issues. A point release was issued just a few months ago to fix one of them, five days (with a weekend in the middle) after receiving the initial report. As usual if y'all find any security bugs in our projects please send an email to security@rust-lang.org.

10 Likes

For varying license terms I recommend cargo-deny, allowing you to blacklist crates and enforce a list of licenses.

1 Like

If you don’t want to use such tools as above, you’ll always be happier with things that are already in the std. Rust makes strong upwards compatibility claims, as such you can expect std functionality to work for a long time on all major platforms. It’s the same as when you use the OS-builtin tools vs 3rd party ones, they’ll probably be around and working for longer.
This is the problem I imagine other people have with rusts 3rd party library ecosystem.
Also C++ et al have some libraries that have just matured for a really long time or become the defacto standard (boost,qt) but are also supported and maintained by the industry. Rust currently just does not have this background and years of service + company backed support.
This will take some time and this can be worked around by directly documenting for example the rust-nursery crates as a go-to, as well as a curated list of crates along with tools like cargo-deny.

2 Likes

I don't think it's fair to say that it's a "completely hassle-free non-issue", when building the simplest app with pretty much any of the popular web frameworks takes many minutes and produces a deps folder hundreds of megabytes in size. It's one of the first things I tried doing when playing around with Rust (with actix, but that seems to depend on tokio, which depends on hyper, or some combination thereof), and it nearly baked my old laptop.

2 Likes

Just wanting to remind that crates.io wants to have pre-built versions in the future. Otherwise yeah: one big downside is my 1.5 GiB debug folder with 2984 files for said project, with a fresh clean all 6 weeks. Rust will kill my SSD someday :stuck_out_tongue:

Please lay off the trade-snark. It implies that you're using "enterprise" in a non-literal sense, and the non-literal senses of the word "enterprise" are usually bad. You don't know that their code is bad, since it's hypothetical, meaning you can't possibly have read it. And since you're talking primarily about people who're working under IT departments that restrict dependencies, it seems like you're implying that anyone who works under such an IT department automatically produces overengineered enterprise™ software; an implication that is extremely uncalled-for.

Since it's part of a larger phrase, I assume you didn't mean it in the "Enterprise FizzBuzz" sense, but rather in the "expensive" sense, but it seems unsafe to assume that everyone's going to understand it the way you want them to. Which is why I'm making the specific recommendation: don't use snark on the internet. It will be misread, and I'll wind up getting your posts in modmail when someone misreads it. The fact that you've flat out called people's lived scenarios "not legitimate" isn't going to help coax them into a charitable reading, either.

8 Likes

Moderator note: Official Rust spaces are places where people are nice to each other, in line with the CoC. People who can’t follow these minimal standards of decorum may find themselves excluded, regardless of any technical arguments you may be making (and certainly regardless of whether the mods agree with your technical viewpoints). If you have questions about moderation, then please email the mods: rust-mods@rust-lang.org.

8 Likes

Though I don't completely deny a small stdlib, this statement just doesn't make any sense, unless the lib itself is something nobody going to use of course.

The live example here is Go, which has a huge stdlib, and those libs receives regular maintenance by community and Go team. Similar thing is happening to Java, Python, PHP as well.

I don't know how this stdlib == dead libraries thing got into people's mind. A third-party lib is much more likely to die than the one in stdlib. "Lost of interest", "Lack of time", "No successor maintainer" to name a few reasons.

Do you depend on a particular lib for your enterprise™ multi million dollar operation? Fork it

If every company has to do something like that, then they may as well build a language/lib by themselves. Why bother trying out a "untested" new one? OR, they may just use the language that provides the complete package they needed. There is a reason for Java EE to exist and grow to that scale.

Language development should always facing the market demand, not just "We are building something cool which features X Y Z". Nobody is going to use your language if it can not help them make money, quickly. Welcome to the Real World Co., where "untested" and "new" is Risky Factor™.

No, all experience has shown the std is what goes unmaintained

Is this throw-away gigs? I can provide example where security issues is fixed right after discovery.

I'm actually agreed with many view points which supports a small stdlib. It certainly got their own advantage (and of course, disadvantage). Admit those cons and pros will make the community more heather as it makes the goal more clearer. Blindly denying it won't make the language better, it will only make the community toxic.

2 Likes

I take the statement "std is where libraries go to die" to really mean that std is where generally (as in popularly) useful libraries go when they are fully baked. Nobody is going to make a faster version, or come up with a groundbreakingly more expressive API. We know how to do this, everybody needs it, there's no point in having multiple competing versions of it. Yes there are exceptions to all of that (some of them quite spectacular), but that is what it means for a lib to "die" in this context.

3 Likes