Proto-RFC: "Approve" of std::fmt::Write for general-purpose use

[SimonSapin]

Background

So I have lots of API that build up strings piece by piece. The easiest way to do it is to return a String. For example, url::form_urlencoded::serialize does this.

One issue with this approach appears when one wants to compose such functions. You might want to have, say, a form-urlencoded query string in the middle of an URL. Intermediate String return values each have their own memory allocation, which is wasteful.

So instead of a returning String, a function can take a &mut String parameter and append to it. I do this in e.g. url::percent_encoding::percent_encode_to.

Finally, what if the ultimate goal is not to obtain a single string in memory but write text to a file, or push it through a pipeline that processes it incrementally? The String::push_str method can be abstracted into a trait, and API generating text be made generic over implementations of that trait. This is similar to the std::io::Write trait, but with Unicode strings instead of bytes. I’ve made a TextWriter trait for this. (See it used in e.g. cssparser::serializer.)

std::fmt::Write

The formatting system (behind the format! macro and the Display trait) has exactly the same requirements, and the std::fmt::Write trait was introduced for the same reasons. It is almost identical to TextWriter, except that the documentation discourage its use:

This is similar to the standard library’s io::Write trait, but it is only intended for use in libcore.

This trait should generally not be implemented by consumers of the standard library. The write! macro accepts an instance of io::Write, and the io::Write trait is favored over implementing this trait.

Proposals

I’d like to propose three changes. But the conventions here are more important than the actual code changes.

1. “Approve” of using std::fmt::Write other than for the formatting system by removing the quoted bits of documentation. It feels silly to maintain a trait almost identical to one in the standard library just because of a convention not to use the latter.
2. To further show that they are not necessarily linked, move the trait out of std::fmt. I’m not sure what module is a good destination, maybe std::str? Edit: Not std::io, I think. I/O is really bytes.
3. Optional: Add a write_char method to the trait, with a default implementation based on char::encode_utf8 and the write_str method. This is not strictly necessary (one can always use write!(w, "{}", c)), but I don’t see a reason not to have it. char is on of Rust’s primitive types.

CC: @aturon, @alexcrichton.

[alexcrichton]

I’d be fine with rewording documentation and adding write_char, but I would want to think more before moving it out of std::fmt.

[mzabaluev]

Of relevance: Pre-RFC: Separate reading/writing String from std::io::Read/std::io::Write

I’m fine with keeping the trait path as it is, but std::fmt::Error should expose a possible underlying std::io::Error.

[mzabaluev]

A possibly important difference is lack of flush in std::fmt::Write. This may be understood as a feature to make general-purpose formatted output as simple and flexible as possible (e.g. stdout/stderr may flush automatically on newlines), while flush may be provided by other traits or simply type-specific methods. But the case of a “strings-only writer with flush” should be possible to express in trait bounds.

[aturon]

I’m on the same page as @alexcrichton – I’m wary only about moving it.

The main reason I’m hesitant to move it is that I’d like to be very confident that this is the TextWriter trait we want before we stabilize it in a “good” location. Simon, you’ve given this a lot more thought than I have, and if you are reasonably satisfied that this covers our needs there, I’m open to it.

@alexcrichton, I assume this has to be stable for our various formatting macros to work?

[alexcrichton]

@alexcrichton, I assume this has to be stable for our various formatting macros to work?

I think actually we could get by without the trait being stable. This means that you could not call the fmt::write function but that’s pretty rare anyway. Most interactions with std::fmt are either through fmt::Formatter (stable) or std::io::Write::write_fmt (stable).

So de-stabilising this trait may actually be a good idea.

[SimonSapin]

I’m on the same page as @alexcrichton – I’m wary only about moving it.

Ok, fair enough. I can also start using it where it is.

Simon, you’ve given this a lot more thought than I have, and if you are reasonably satisfied that this covers our needs there, I’m open to it.

So far I’ve mostly used and wanted to use this for serializing things to a given file format. This is only some of the use cases, so I can’t really talk about all of everyone’s use cases.

This said I’m reasonably confident about they write_str and write_fmt methods as they exist today − there just isn’t that many possible ways to design them. We can always add more features (including more info in fmt::Error) later backward-compatibly, right?

So de-stabilising this trait may actually be a good idea.

This would defeat the point of saying it’s OK to use this trait outside of the formatting system, if we want most of the ecosystem to use stable Rust by March 9.

[aturon]

This would defeat the point of saying it’s OK to use this trait outside of the formatting system, if we want most of the ecosystem to use stable Rust by March 9.

So, the main goals of “blessing” a trait in std would be to (1) enable interop between various libraries and (2) provide basic building blocks in std itself.

Going forward, our hope is to gain experience and buy-in with abstractions in crates.io first, and then move things into std as they are becoming de facto standards (or in some cases to help move things in that direction).

I think the idea of destabilizing fmt::Write is not to encourage people to use it there (which would require using nightlies), but instead to allow a separate version of the trait (and other infrastructure) to be developed in crates.io and then later replace fmt::Write if/when we move it into std. Whereas if we ship a stable trait now, we won’t be able to iterate as much on its design.

@SimonSapin, what do you think of exploring this space and gaining buy-in in crates.io first, and leaving room to replace fmt::Write with the resulting design later on?

[SimonSapin]

https://crates.io/crates/text_writer has existed for a while, and this thread was to discuss whether we have reached that “later” point. Though admittedly I’m not aware of code I’m not involved with using this crate.

[mzabaluev]

I’m considering using TextWriter for the yet-to-be-written encoder part of encoding-io.