Managing libstd's dependency on libc

[davidc]

While investigating a recently-opened PR for the serial crate, I realized that the relationship between libc and the standard library is a dependency that can't be accounted for with Cargo's current dependency management mechanisms. In searching for previous discussion on this topic, I found that this was left as an unresolved question in RFC 1291:

What is the relation between std::os::*::raw and libc? Given that the standard library will probably always depend on an in-tree copy of the libc crate, should libc define its own in this case, have the standard library reexport, and then the out-of-tree libc reexports the standard library?

Right now, the standard library exports types from an in-tree copy of libc, which opens the possibility of having an incompatible version of libc declared as a dependency in Cargo.toml. I can illustrate this by compiling a very simple program for ARM with Rust 1.5 and Rust 1.6:

extern crate libc;
use std::ffi::CString;

fn main() {
    let string = CString::new("Hello, world!").unwrap();
    unsafe {
        libc::puts(string.as_ptr());
    }
}

• CString::as_ptr() returns *const c_char (c_char defined by the standard library).
• libc::puts() expects *const c_char (c_char defined by libc).

The results of compiling this for arm-unknown-linux-gnueabihf with different versions of Rust and libc are as follows:

• libc v0.1, Rust 1.5: Success
• libc v0.1, Rust 1.6: expected *const i8, got *const u8
• libc v0.2, Rust 1.5: expected *const u8, got *const i8
• libc v0.2, Rust 1.6: Success

This shows that each version of Rust is tied to a version of libc, and it's possible to get compilation errors by using a different version of libc than what was used to compile the standard library. This was also pointed out by cuviper a couple of months ago after it affected the ncurses crate:

But it's a tricky issue that you may have different c_char in CStr depending on the compiler version, while the c_char you get from libc depends on which external crate you grab. You can use std::os::raw::c_char to match the builtin libc and CStr, but you need the crate's c_char to call its functions. Hmm...

Now that there are at least two examples of this breaking in the wild, I'd like to re-raise the question that was left unresolved in RFC 1291. How can we manage the relationship between libc and the standard library?

I have a few ideas to contribute:

1. Is it possible to have the standard library provided as a separate crate? That would allow it to declare its dependency on libc in a way that can be resolved by Cargo.
2. If Cargo could allow crates to declare a minimum supported version of Rust, libc v0.2 could have declared that it requires Rust 1.6 or later. That would only prevent the compilation error from upgrading libc, but not the one caused by upgrading Rust.
3. Re-export the types from the standard library in libc. This means the type of c_char doesn't change until upgrading to a new version of Rust. This is the solution mentioned in the RFC, but is probably no longer viable since libc is getting a feature flag for #![no_std]. I think this solution is also weird, because technically, the standard library depends on libc and not the other way around.

In an ideal world, I like the idea of combining solutions #1 and #2 above. That would allow the standard library to declare its dependencies both on Rust language features and on libc. I believe that could have prevented both types of compile errors shown above. libc can get #![no_std], so there's no cyclic dependencies. (Long term, #![no_std] could be just a lack of a declared dependency on a std crate.) I say in an "ideal" world because I'm guessing that separating the standard library from the compiler would be a huge undertaking due to feature flags and stability attributes that aren't available in regular crates.

What are other people's thoughts on this issue? Are there other ways to manage the relationship between libc and the standard library?

[steveklabnik]

I am hoping that https://github.com/rust-lang/rust/pull/31123 is the first step towards rustc just using the same crates as everyone else, which would alleviate this.

[davidc]

Thanks @steveklabnik. I just read through that PR. It looks very promising, and I can’t wait to see what comes out of it. If the outcome is that the standard library becomes a first-class crate, that would awesome.