Redox support in `liblibc` and `libstd`

[brson]

Out of curiosity can you procure a GitHub link diffing the two projects? I’d like to see how much code is involved.

If we decide to merge support for the two I don’t think an RFC will be required. We routinely add platform support.

The main issues here in my opinion are:

• The size of the patches. If the amount of code is comparable to a unix that is easier to consider than if the amount of comparable to the difference between unix/windows.

• The amount of maintenance burden this will impose. Redox is under heavy development. I don’t want to see the rust-lang/rust PR queue overwhelmed by Redox churn. Rust contributors shouldn’t have to deal with that.

• Not all OSes can have in-tree support from Rust. std is intended to be decomposed in a way that allows others to build on that work out-of-tree, though it does not fulfill that yet. That you are currently successfully overriding liblibc is great - that’s exactly how it should work. std itself though needs a great deal of refactoring to support this. I have previously outlined how I believe std should be architected to allow std to be reused out-of-tree for other OSes. As a high-profile, and active Rust project, Redox may be appropriate to support in-tree, but that’s not obvious; there will definitely be a lot of push back.

[jackpot51]

A warning, I have not tried to minimize the changes yet. Go the the files changed tab to make it easier to view.

Liblibc: https://github.com/rust-lang/libc/compare/master...redox-os:master Libstd: https://github.com/rust-lang/rust/compare/master...redox-os:redox

[brson]

Thanks @jackpot51. The diffs are encouraging. Changes are mostly isolated to their own submodules.

Are there significant parts of std yet to be implemented or that you expect to change? Does redox modify any of the other std crates besides these two?

[jackpot51]

There are advanced networking functions yet to be implemented, multicast and broadcast for example.

No other crates have needed changes yet.

[jdub]

A write-up of this process (perhaps written as documentation, or something that can be massaged into documentation) would be super useful for future porters.

[brson]

I haven’t forgotten about this thread, I’m just pretty torn about how to proceed. My conflicting thoughts on this:

• I want to support Redox to the extent that we can, while also balancing what’s appropriate for upstream Rust
• I want it to be easy for experimental projects, OS and otherwise, to reuse the standard library and other parts of the main Rust codebase
• I realize that maintaining patches out of tree sucks
• I don’t want mainline Rust to undertake maintenance of code to support every Rust OS project because of increased maintenance burden. Having such a blanket policy seems likely to cause sadness for the Rust maintainers.
• I think there is a very plausible story for using crate replacement to abstract the relatively small platform-specific components of std, and we’re pretty close to making that possible now. This is how I want out-of-tree std porting to work.
• In lieu of having that story in place I think there is a reasonable argument that mainline Rust should be willing to have support in-tree for projects actively maintaining ports of std, so that they don’t have to keep forward porting their patches, assuming those platform-specific parts are cleanly factored out of the primary code paths. I see this as somewhat similar to the Linux kernel begging vendors to put their niche drivers in tree.

I’ve put this on the libs team’s agenda for some high bandwidth feedback but it hasn’t happened yet. Maybe some could give an opinion. cc @library_subteam.

[alexcrichton]

I agree with @brson that so long as patches are minimal and non-invasive it’s nearly free for us and we could land at any time. If the changes needed are quite extensive, however, or have a lot of churn, then we may want to hold off for now or find a different solution.

[brson]

The libs team agrees that we can support redox and other such platforms in-tree, as tier-3 platforms, with the following conditions:

• The port must support more-or-less the entire std API. This is sort of the minimum barrier to entry. Today we don’t have any platforms that conditionally-compile out parts of std. It looks like redox has problems with Path, which is very Unix/Windows-specific, and that’s a problem we need to solve. The emscripten port does really poorly here though - it does build the entire API surface, just most of the I/O is broken at runtime today.
• The platform-specific parts need to be limited to a few subsystems, namely std::sys, libc, the alloc and unwind crates. Today that is not quite true, with std containing a few platform-specific cfgs scattered around, but I started a patch to enforce this rule. This is to keep the maintenance burden down, enforce abstraction boundaries that might one day allow us to extract sys from std.
• We are not going to offer automated regression testing, nor binary builds, nor guarantee the build doesn’t break. Inclusion of tier-3 platforms is on a best-effort basis.
• We reserve the right to move teir-3 platforms back out of the tree if we ever reach a solution that allows ports to be maintained easily out of tree.
• This is still not a blanket policy that all such platforms can have in-tree support; they’ll be considered case-by-case.

So next steps are to submit a PR for further review.

[jackpot51]

The port supports the majority of the std API. The issue you have linked is not because of missing Path functionality, but because Redox sets cfg(redox) instead of cfg(unix) or cfg(windows) and the associated code unnecessarily assumes that one or the other is set. Unlike emscripten, the majority of functionality is implemented and usable.

[jdub]

The port must support more-or-less the entire std API.

Please consider expanding on this policy to cover niche platform support in core and other sub-std (sorry) crates. For example, although the Cortex-Mx RFC concluded that only the targets would be added for now, I imagine it and other platforms will ask for in-tree support relatively soon.

Thinking a little further along, what about platforms that could support all of std except, say, file I/O?

[jackpot51]

I have done the first patch here: https://github.com/rust-lang/libc/pull/439. It is as minimal as possible and is designed to allow programs that naively import libc to compile.

[jackpot51]

Is there a place in librustc* where cfg(unix) and cfg(windows) are defined? I need to either add cfg(redox) or use cfg(target_os = “redox”) instead

[japaric]

use cfg(target_os = “redox”)

This should Just Work without modifying librustc*. You just need to set target_os: "redox" in your target specification (the .rs file in src/librustc_back/target).

[jackpot51]

It was pretty hard to find but here it is: https://github.com/rust-lang/rust/blob/master/src/librustc/session/config.rs#L959

Should I PR a change to add | “redox” ? Otherwise, I need https://github.com/rust-lang-nursery/rustc-serialize/pull/170 merged

[eddyb]

unix and windows correspond to target_family, of which redox is unix. You should use target_os, at least for the forseeable future.

[jackpot51]

That is not correct, Redox is not Unix - it has it’s own target_family set to redox and the implementation in libstd will not be done inside of sys::unix, due to fundamental differences (no dependency on libc, for example)

[eddyb]

Ah, my bad then. I would still not modify the compiler unless/until it becomes a first-tier platform.

[jackpot51]

Sure, I will avoid modifying the compiler then. Thanks for the input

[jackpot51]

@alexcrichton @brson @eddyb I have finally make sys::redox compile and boot when using the default libstd on Redox. Here is a PR, meant for discussion purposes, showing the changes: https://github.com/rust-lang/rust/pull/37702

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.