A plan to move most of `std::io` to `alloc`

libs
1

It would be great to have some part of std::io available in #![no_std] contexts, especially the core traits (Read, Write, etc).

Why alloc and not core?

Many parts of std::io explicitly rely on allocation: Error::new, Read::read_to_end, BufReader::with_capacity,... Migrating to a context where alloc is not available would require changing the items themselves, which is not acceptable ; or at least requires significantly more work.

What would be moved to alloc::io?

Everything possible. Once the first difficult points are solved, the rest should follow easily. Of course, everything directly related to the OS (stdio, pipes, IsTerminal) would remain std-only.

The plan

This plan relies on externally implementable items (EII) (Tracking Issue for externally implementable items · Issue #125418 · rust-lang/rust · GitHub), whose implementation is still a draft at this point. The main idea is to use unstable ones so alloc can declare and use them, and std can implement them. Maybe we could get a first version earlier by doing an equivalent in the compiler directly, as for panic_handler and global_allocator, but this would require more work.

  • Notably, alloc would require EII to implement errno: last_os_error, errno classification, and errno messages. A default impl in alloc would return 0, ErrorKind::Uncategorized and "".
  • The other (but hidden) offender is copy(), which has a specialization path depending on the OS (to enable stuff like using sendfile where possible). Once again an EII can solve that.

The harder part is for types depending on the OS. There are two of them here:

  • IoSlice and IoSliceMut, which are guaranteed to be compatible with their OS equivalent. Fortunately, the same layout is used on all existing OS: a pointeur and a usize. If a new OS is added with another layout for their readv/writev equivalent, well, too bad for them.
  • The harder one is RawOsError. This is a type alias to the raw OS error type, which happen to always be i32... except on UEFI. I see two workarounds here:
    • Make a breaking change to UEFI target. I don't know how unacceptable it is, nor if using usize is actually necessary or not. (cc @nicholasbishop @dvdhrm)
    • Accept having a cfg(target_os) in alloc. This small violation is worth if it means having io available in alloc IMO.

The last usage of std::sys is for DEFAULT_BUF_SIZE which is smaller for "espidf" target. We could normalize it if acceptable or use another cfg(target_os) here.

All of this seems really doable to me once EII is implemented.

Links

2 Likes
2

Default impls are the hardest part of EII to implement. Several targets don't support weak symbols at all or have rather buggy support. This includes the windows-gnu targets where weak symbols are non-functional and windows-msvc where exporting weak symbols from dylibs requires a hacky implementation.

1 Like
3

I think we could manage to move a fair bit of io to alloc or even to core, by relying on the fact that all of core/alloc/std are one coherence domain. So, for instance, std::io::Error could move to core while leaving the bits that can work with a boxed user-provided error in alloc and the bits like last_os_error in std.

3 Likes
4

I also wonder how much most of the code involved in these traits really care what the error types are? For example, could we have a GenericRead with an associated Error type, so that std's Read is just trait Read : super GenericRead<Error = io::Error> {}?

It might be easier to get "implementable trait aliases" (or some other form of "your existing impls still work") than to deal with the externally-implementable implementation complexities.

Also, I haven't thought that hard about associated error type vs generic error type here. Maybe generic would be fine since we could have BufRead implement Read for a bunch of different possible error types?

2 Likes
5

Maybe it should be an io crate? With an optional alloc feature?

It would be great to move these crates more towards a regular crates-io crate model and less of built-in magic.

core and alloc end up being "std lite" crates, but different platforms/targets have different capabilities that aren't a strictly linear progression from core to alloc to std.

8 Likes
6

The IO traits are useful abstractions, regardless of platform capabilities. You can use them on top of a slice or Vec without doing any actual IO. They are almost entirely platform independent (with the minor exceptions pointed out in the OP).

7

The std::error::Error got split to be possible to expose it in core. Maybe io::Read/io::Write could get a similar treatment?

read_to_end/_string and _vectored methods aren't necessary for the trait to be useful.

io::Error already supports multiple internal representations. The custom Box<dyn Error> of course wouldn't be possible, but it has simpler ones with os error codes, ErrorKind and &'static str message.

4 Likes
8

Maybe &'static dyn Error may also be useful, especially when errors lead to crash?