Should writeln! expand to write_all when it can?

[glandium]

Currently, writeln! always expands to write_fmt, but in the most simple cases, write_all would be better. (Same applies for println!, although the lack of buffering probably wouldn't make it as visible)

Compare:

use std::io::{BufWriter, Write};

fn main() {
    let out = std::io::stdout();
    let mut out = BufWriter::new(out.lock());
    for _ in 0..100_000_000 {
        writeln!(&mut out, "Hello, world!").unwrap();
    }
}

$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:02 [ 611MiB/s] [    <=>                                                               ]

use std::io::{BufWriter, Write};

fn main() {
    let out = std::io::stdout();
    let mut out = BufWriter::new(out.lock());
    for _ in 0..100_000_000 {
        out.write_all("Hello, world!\n".as_bytes()).unwrap();
    }
}

$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:00 [1.58GiB/s] [ <=>                                                                  ]

LTO does save things a little because the call to write_fmt can be further inlined and optimized, but just calling write_all is still faster:

$ cat >> Cargo.toml <<EOF
[profile.release]
lto = true
EOF
$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:01 [1.12GiB/s] [  <=>                                                                 ]

LTO makes the write_all version faster too:

$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:00 [1.89GiB/s] [ <=>                                                                  ]

[H2CO3]

I don't think this is possible. fmt::Write doesn't have write_all(), and formatting macros need to support fmt::Write as well as io::Write.

[glandium]

Oh right, there's that... but io::Write could have write_str.

[glandium]

This makes me wonder why there isn't a impl fmt::Write for W where W: io::Write... maybe because that would make write_fmt ambiguous?

[cuviper]

A blanket implementation of a trait has to be in the same crate as its definition, but these are core::fmt::Write vs. std::io::Write.

[kornel]

I'd love write! and format! to be smarter.

However, last time this was discussed (sorry, can't find thread), the answer was that these are just macros, and don't have type information. They could be made magically type-aware in the compiler, but that would make them different from all other macros.

[Tom-Phinney]

I'd prefer that they stay as pseudo-macros, even to the point that it be possible to publish proc_macro equivalents (now that proc_macros are stabilized). Thus this is a plea for less magic and more regularity in Rust, not the opposite.

[jhpratt]

I know last time it came up, I considered the idea of a community group to work on proc macros that were significantly smarter. Unfortunately classes started this past week, so I don't have nearly as much time any more. However, I'd still be down for overseeing such a group if desired.

IIRC what came up last time was even things like format!("{}{}", x, y), which could be slightly optimized to use the known lengths of x and y.

[CAD97]

As far as I recall, the formatting machinery included in std works, but basically nobody actually understands how it works entirely. Last I saw consensus was roughly that it'd be simpler to design a new solution (that fits the existing stable API) than to try to incrementally get the existing solution to the desired better end state (at least a portion of which would be allowing switching between fully-runtime-optimized or size-consious approaches.

(Also happy cake day @jhpratt)

[tspiteri]

@Tom-Phinney

Thus this is a plea for less magic and more regularity in Rust, not the opposite.

I'm not sure I see the value in this. The standard library is allowed to do things that cannot be done outside the standard library. And in the original topic suggestion, the interface itself is unchanged. I do not agree with the reasoning "An external crate cannot be optimized this way, so we should not optimize the standard library this way." Of course there could be other reasons not to do this, for example the optimization's implementation might be too hairy and not worth the development/maintenance cost. But if something in the standard library can be optimized with internal magic and the cost/benefit ratio is in favour of the optimization, I think it should be.

[jhpratt]

Yeah, it may have been that. Or both, I don't particularly remember that thread well. I just know there were some shortcomings that people had an interest in fixing.

[Tom-Phinney]

Philosophically I agree on the optimization aspect of specialized compiler support, but not on the aspect that the compiler magically implements things in a pseudo-macro that are impossible for others to implement in a real macro. For example, if the compiler uses type knowledge derived from post-macro-expansion type inference in its implementation, then I would prefer to see Rust extended to permit such capabilities added in a new form of later-compiler-phase-executing macros that have access to that same information.

rustc's unstable plugin capabilities do provide means to achieve this, and more, but those plugins are for a specific implementation of the Rust compiler, so are unlikely to work with alternate-source compilers that will come into existence as Rust matures. In fact existing plugins are rustc-version specific, so sometimes require updating when new rustc versions are released.

[pcpthm]

This kind of ad-hoc polynomrphism can be done using the autoref-specialization trick without compiler magic: Playground

struct IoWriteKind;
impl IoWriteKind {
    fn write_str<W: ?Sized + std::io::Write>(self, w: &mut W, s: &str) -> std::io::Result<()> {
        w.write_all(s.as_bytes())
    }
}

struct FmtWriteKind;
impl FmtWriteKind {
    fn write_str<W: ?Sized + std::fmt::Write>(self, w: &mut W, s: &str) -> std::fmt::Result {
        w.write_str(s)
    }
}

trait IoWrite {
    fn write_kind(&self) -> IoWriteKind { IoWriteKind }
}

trait FmtWrite {
    fn write_kind(&self) -> FmtWriteKind { FmtWriteKind }
}

impl<T: ?Sized + std::io::Write> IoWrite for &T {}
impl<T: ?Sized + std::fmt::Write> FmtWrite for T {}

macro_rules! my_write {
    ($out:expr, $str:literal) => {
        (&$out).write_kind().write_str($out, $str)
    }
}

fn main() {
    {
        let mut out = std::io::stdout();
        my_write!(&mut out, "Hello, world!").unwrap();
    }
    {
        let mut out = String::new();
        my_write!(&mut out, "Hello, world!").unwrap();
        assert_eq!(out, "Hello, world!");
    }
}