Refactoring std for ultimate portability


Thanks @jethrogb. The main difference to my idea is, that my idea is not bound to traits and doesn’t use impl. With this you can use it for parts, requiring structs. E.g. defining that there has to be a struct PathBuf which’s (non trait) impl has a method fn new() -> PathBuf etc. Also that std::path::PathBuf should be a valid path to access it.


is there a way, to make symbols public wrt. to building std. But private wrt. to using it. It probably don’t work when rust parts of std are dynamically liked in, but if a single, possible dynamic linked in, std artifact is produced it could work, or? (I’m not saying we should do it, I just wonder if we could do it if it is needed)


When building a shared object you can choose which symbols to expose. So yes, if you were to build the entire runtime into a single shared object a lot of internal symbols would disappear. However, you’d of course lose all the pluggability you’d gain from using separate objects.


Could this functionality be provided by an annotation, using the #[] syntax? Maybe like #[provides(item.path)] for the implementation and #[providable] for the definition.

Maybe I misunderstand those annotations (I don’t know what they’re called in Rust).


I think so, you could see #[provides(item.path)] as some how analog to #[lang="some_lang_item"] but more general. An use #[providable] on a struct, struct+impl or trait. (through there might be some corner cases)

I will move this part of the discussion to the mentioned pre-rfc :smile:

Also while I think that this kind of mechanic can be a grate help for std and other bigger libraries where the logical seperation of components is not allways on the same plane as the os-specific/unspecific seperation, I don’t know if it would be good for the general rust language.


I think per function providables are a good idea.


@brson I’d like to keep moving on this. What would be a path forward on this? Reading this thread there weren’t really any objections to your proposal. Does someone just need to do the work to implement your proposal? Do we need to do some more design first? Do we need an RFC?


Let’s do this so we can more easily add CloudABI to Rust :slight_smile:


@brson another ping now you’re back from your conference


@jethrogb Hi, I’m so glad you are interested, and I’m sorry for not following up sooner. I don’t think there are any obstacles to proceeding, as refactoring in this direction seems fairly unobjectionable.

There are a few ways to proceed, but I think getting to the point where you can implement a custom std out of tree will be a long road.

For refactoring std, the easiest thing to get started is by cutting down entries in the tidy whitelist, where std is not using conditions in the right modules. These are easy patches to make and review.

Someone could also be bold and create the pal_common / pal_unix / pal_windows crates and wire them up to the build system. I suspect that would get past review with some debate. After that there will be a long slog of puzzling out how to pull pieces out. I mentioned some other next steps in my op. And remember my linked branch has uncommitted refactoring steps that, while bitrotted, demonstrate some sequences that work to untangle dependencies.

As far as your end goal of creating a std port that excludes large chunks of std, for now I would focus on creating a port that fits into the model described here, where the customization is at the sys/pal layer. Today, I would suggest that, instead of trying to compile out parts of std your port doesn’t use, settle for panicking instead. In the longer term, when std has scenarios, then we can actually remove the unsupported features. This might involve a custom platform under the libstd/sys directory, and ultimately a custom pal crate.


@jethrogb Any progress? I’d love to help.


Nothing yet and nothing planned for the next month and a half at least, so feel free to start anywhere you want.


I’ve been working on something the last few days. I started with std::fs, which was rather trivial to create an abstract layer for (since it’s already pretty cleanly seperated) and I’m currently shuffling around std::net. Would love to hear some thoughts about it.

edit: Whoops, forgot the link:


Neat! I looked over the diff so far and it all looks reasonable to me. The main question I have that you might have a better look at since you’ve touched the code: does this actually reduce the interdependencies such that it could be separated out into a different crate?


Yes! I intend to make ::pal an external crate at some point. Interdependencies are supposed to expressed by using associated types on the trait (possibly reusing traits for other modules). ::pal currently depends on ::io::Result and ::time::SystemType, but the former can be moved to ::pal and reexported by libstd and the latter probably will find it’s place as abstraction in ::pal::time.

jethrogb schrieb am Mi., 26. Juli 2017, 22:54:


What about things like std::path?


I haven’t looked at std::path yet, but I assume that it should be no different than the other modules.


impl Trait in structs would be nice to avoid accidentally calling non-trait methods (especially inherent methods with the same name are a risk)


On that note, would all these traits and associated types cause error messages to blow up in complexity for the end-user?


The end user? Users of the libstd won’t see any of that and implementors of the backends would at most see a bunch of “this function is missing in this impl for SomeTrait” once per missing function instead of everytime per use of a function. Also, the error messages are generally much friendlier since they’ll also include the types.