The Libs Team Mission

[jethrogb]

For what I’m using rust for, game dev, I’d like to see libs that are no_std whenever possible

Could you elaborate on this? Why can’t you use std for game dev?

[Centril]

Very Interesting post. Thank you for writing it!

I have somewhat a fear that the libs team doing too much review of the crates ecosystem (which is of course great) might leave too little time for the standard library which is vital to everyone. Do you share this view?

Also; What are, in your view, the reasons for and against having an extra ‘ecosystem team’ that is dedicated to working on crates, and an std team that works on the standard library (libstd, libcore, liballoc, and other nursery crates)?

[aturon]

Great questions, @Centril!

I have somewhat a fear that the libs team doing too much review of the crates ecosystem (which is of course great) might leave too little time for the standard library which is vital to everyone. Do you share this view?

I understand where this worry is coming from, but part of the strategy here is to substantially expand the team – in part by decoupling it from std, so that we can have a smaller, dedicated working group for focusing on std. IOW, if this goes well, I anticipate better maintenance for std, because we’ll have a tigher group that’s more intentionally focused on it.

What are, in your view, the reasons for and against having an extra ‘ecosystem team’ that is dedicated to working on crates, and an std team that works on the standard library (libstd, libcore, liballoc, and other nursery crates)?

That’s kind of what I’m proposing, with two differences:

• First, it’s a working group within the libs team, which ensures some level of coordination with activities around std and work like the API guidelines.
• Second, I think that having a single monolithic “ecosystem team” won’t work well; it’s too vast. In the past, when we’ve had teams with that large of a scope, things don’t function well because any given topic ends up not being that relevant to a good chunk of the people involved. That’s why I think it’s better to spin up more topic-focused working groups (SIMD, networking, concurrency, …)

[sfackler]

Improving the -sys crate situation is one of the things I’m the most interested in working on this year!

[susurrus]

The Lib Blitz was a really exciting time seeing advancements in some crates and some hit 1.0, but the biggest take away I saw from it were the API guidelines that were published. I’m working through them in order to get nix to 1.0, and they’re full of very sage advice. I’d like to see efforts in the Libs Team expanding this knowledge to other areas related to developing great crates like:

• Example CI configurations for some of the bigger CI players, especially as this relates to cross-platform compilation and testing (like using trust). I’d especially like if this includes GitLab CI, which generally gets neglected with these kinds of things. The crates deepest in the dependency tree are most important to have cross-platform CI testing as possible. We wouldn’t support as many platforms as we do with nix if libc’s CI testing wasn’t so robust.
• This also means getting all supported platforms up to being supported in rustup. That’s the only things stopping us from supporting DragonFlyBSD in nix right now, that we can’t install it easily (same for OpenBSD I believe). It’d be good if there was better documentation and support around how to set up compilation and/or testing for other platforms either using QEMU, docker containers, etc. There’s knowledge hidden across a lot of different crates, but it’d be good to centralize that knowledge so everyone can benefit.
• Clippy and rustc’s lints are very powerful for enforcing good code style. gtk-rs and winapi-rs go beyond these lints to enforce additional code style rules that accelerate development. It’d be good to see more effort towards adding more lints to rustc and clippy and to also enable more of them by default.
• More validation of crates before uploading to crates.io. Right now a lot of failures when publishing to crates.io happens really late in the process, but it’d be great if the API guidelines above was an automated check run my cargo or a cargo subcommand. The easier the guidelines process is, the more likely people will try to follow it.

Regarding specific libraries I’d like to see some focus on the bottom-of-the-stack: platform libraries. This means winapi-rs, libc, mach/CoreFoundation-sys/IOKit-sys, ioctl-rs, and nix (note: I maintain this crate). These are all very low-level, in that they interact with the system kernel or user space libraries associated with the OS. It’s very hard to build software on an unstable foundation and these libraries are very foundational. The biggest issue is that libc needs a 0.3 release to fix various issues with it, but nobody wants to touch it. That’s the biggest wart I see, but many of these crates need work to build off the progress that’s been made in the ecosystem, like relying on libc because of its good CI testing and validation rather then declaring their own types. It would be helpful to get enough eyes on these foundational OS crates to set them on a path to 1.0 as part of a lib blitz, and maybe help them recruit some more devs to do that work.

[Lokathor]

(I speak as a hobbyist who watches a lot of game dev talks and tinkers with stuff in his garage but who doesn’t work in the AAA space or anything like that)

The short answer is that you can use std for some game dev, but not all of it.

[Thiez]

Could you elaborate on that? Are you talking about games that must run on “weird” platforms (e.g. consoles)? Because apart from that I can’t think of anything where having the standard library around would be a downside.

[Lokathor]

There was a bigger post there, but it got kinda really bigger because I’m often bad at keeping it terse. jethrogb and I decided that such a discussion would easily derail this thread and so I decided to cut it all down to just the short answer there. Fear not, the full text of the post has been saved, and will be morphed into a larger and longer posting about rust that I plan to do at some point.

[derekdreery]

How about a rating system for crates.io, where if you’re logged in via github you can either tick a crate - meaning it did what you needed it to, or question-mark it, where you are saying “I couldn’t figure out how to do my thing”, with a comment explaining your problem.

You could then show metrics like “no. of ticks”.

I think if you have a rating system you want it to emphasize positivity - you don’t want people making unconstructive negative comments. That’s why I don’t think the question mark metrics should be publicly available, but they may be useful to the crate developer!

[steveklabnik]

https://github.com/ErichDonGubler/not-yet-awesome-rust/issues should be useful to the libs team for this kind of thing.

[SimonSapin]

• Example CI configurations for some of the bigger CI players, […]
• More validation of crates before uploading to crates.io.

I don’t know how much any of this fall under the responsibilities of the libs team but I think these two could be tied. I’d love to have CI publish a new version on crates.io and push a git tag automatically after checking that that pass. (For example when crates.io doesn’t have yet a version with the same version number as in the commit being tested.)

[sfackler]

I don’t know how much any of this fall under the responsibilities of the libs team but I think these two could be tied. I’d love to have CI publish a new version on crates.io and push a git tag automatically after checking that that pass. (For example when crates.io doesn’t have yet a version with the same version number as in the commit being tested.)

I think these kinds of things definitely do fall under what the libs team should be doing. It’s a part of what I would like the current API guidelines will grow into - a more fully fledged resource for helping people to make high quality crates.

I have looked into setting up published via CI before, but think we’d ideally grow crates.io’s authentication support out a bit. In particular, I’d like to be able to create auth tokens that are only allowed to publish a single library to minimize risks if credentials leak.

[KodrAus]

I’m excited about the granularity and inclusiveness I think working groups will offer! There are a few things I’m personally interested in pursuing with respect to improve the quality of the crate ecosystem right now.

I’d love to see structured logging in log move forward.

It would be good to try rejuvenate some of our long-time nursery residents and revisit their paths to stabilisation that we identified last year. The nursery doesn’t seem like an ideal long-term home for libraries, and the times where we’ve been able to hand over maintainership to the wider community seems to have worked out well in the past. So maybe that’s a way forward for some?

I’d like to see what we can do to help some of our stable or currently stabilising dependencies flesh out their docs. I think serde sets a great example with its docs + book, but not all libraries would need it.

And I’d like to continue growing our resources like @sfackler mentioned, but in a write by doing fashion, where we support authors’ following resources while producing them.

Anyway, those are my thoughts

[vitiral]

I would like to announce the creation of the ergo crate ecosystem. It is very early stages and I am looking for others who are interested in contributing. All contributions are welcome, from discussion, design goals, maintainers and new crate submissions.

See more about the project here: https://github.com/rust-crates/ergo

I would love to have the libs-team onboard as owners with a voice and decision making power within the ecosystem. Anyone who is interested please let me know!

[serejkus]

One thing I’ve noticed while participating in Impl period is that there is a need for inactivity tracking automation of github pull requests and issues.

Some github issues had comments like “I’d like to take on that”, and that was the last activity on that issue. Sometimes issues like this were retaken and still nothing happened.

There had been another type of inactive issues: someone submitted a pull request, there’s been a discussion, pull rqeuirests has been updated and nothing happened afterwards.

Both of these scenarios could have benefited from pinging automation, like some sort of bot asking “what’s happening?” and changing issue state, like moving it from open - active - review - closed states. Another benefit is that this state may be integrated with impl period/libz blitz list of open issues, so that anyone looking for a way to contribute can checkout open issues only.

[aturon]

I wanted to revisit this topic now that the 2018 roadmap seems to be solidifying.

The roadmap highlights four domains to focus on for 2018, with the hope of presenting a polished end-to-end experience as part of our epoch release (and featured on the web site). The domains are:

• Network services.
• WebAssembly.
• CLI apps.
• Embedded devices.

The roadmap proposes to spin up “working groups” for each of these areas, reporting directly to the core team.

What I wanted to propose was that the libs team work this year also focus on these four areas, in concert with the working groups. This work would go deeper than the Blitz we did last year; the goal would be to bring together domain experts with seasoned library designers and see how we can push things along.

I feel like directing this work specifically toward “what does the end-to-end experience feel like in domain X” will greatly sharpen the effort. But I could use help brainstorming how we might structure it.

wdyt?

cc @KodrAus @vitiral @killercup @withoutboats @mgattozzi @japaric

[KodrAus]

Perhaps we can start by working with the domain experts to rephrase what does the end-to-end experience feel like in domain X? into what do we need for the end-to-end experience of working in domain X to be compelling and productive? so we can identify areas that need attention?

For example, maybe for WebAssembly we could look at what it would take to stabilise stdweb, and for embedded devices we could look at helping library authors make no_std support a first-class consideration.

[vitiral]

First of all, let me just say that I am a just a rust enthusiast who has been writing a CLI application for the past year and a half. I am currently rewriting it using all of the knowledge gained and trying to follow best practices of composable libraries with standardized APIs. It is for this “usecase” that I wrote the blogpost which was linked in the roadmap as well as started the ergo crates ecosystem project which myself, @killercup and @llogiq are now admins on. I have just released an initial alpha for the crate which I hope to use to iron out details in the comming month.

I have not yet been involved in the libz blitz (I helped out with Incremental Compilation for the impl period). However, I am extremely excited by the reception the blog post and ergo ecosystem effort have received and have an immense desire to make this a community effort. I am currently reviewing the API Guidelines and would love to be a part of the libz blitz effort if they would have me. I hope to make ergo much bigger than myself, with the goal of improving CLI ergonomics for the community as a whole, with the community as the primary decision makers on the API we want.

The rust ecosystem is already extremely ergonomic, the main things that need to improve are accessiblity and API simplicity. Ergo aims to make programming in rust at least as ergonomic as programming in any other language. I think this is a very achievable goal in 2018 using the strategy outlined in the ergo roadmap.

[mgattozzi]

I agree with @KodrAus here, but I wanted to expand on it a bit more as well as answer your original question. I think we have two things we want to consider: Where do we want to go and how do we get there?

I think we can have long term goals beyond 2018 for these fields but we only have so much we can do in a year. I think we should first define what we want to accomplish in these areas within the year. This makes it easier for us to then define how to get there. Do we need to stabilize language features (probably) or is there something we as the community can do to help polish up crates critical to the ecosystem? Maybe we need to pull them in as part of official tooling like xargo.

Importantly though we will want to define what the real overarching goal is for those domains. If in the case of wasm we want it to run everywhere then we might better spend our efforts on improving the wasm target in the compiler than working on JS Interop.

I think if we do the following we can really push the vision you want:

1. Define what our vision for that domain is (e.g. Make it easy to make CLI apps or make embedded devices support the same level and quality as Tier 1 systems)
2. Define the problems we want to solve in the domain given our time that support those goals
3. Figure out what it would take to solve that problem (compiler support? crates polished in the ecosystem?)
4. Gather people to work on those problems, clearly define them, write out mentoring guidelines as well for others to contribute who are not in the WG or Libs teams, and take a crack at them.

I think the hardest part will be defining just what we want to do in these areas. It’s nebulous at best. We might want to ask users in those domains what their pain points are, aggregate that data, and use that to direct our decisions.

[aturon]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.