# Should writeln! expand to write\_all when it can?

**URL:** <https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997>\
**Category:** Uncategorized\
**Created:** [September 4, 2020, 12:50am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997 "2020-09-04T00:50:06Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![glandium](https://avatars.discourse-cdn.com/v4/letter/g/3d9bf3/32.png) [@glandium](https://internals.rust-lang.org/u/glandium)\
**Post date:** [September 4, 2020, 12:50am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/1 "2020-09-04T00:50:06Z")

</div>

Currently, writeln! always expands to write\_fmt, but in the most simple cases, write\_all would be better. (Same applies for println!, although the lack of buffering probably wouldn't make it as visible)

Compare:

```rust
use std::io::{BufWriter, Write};

fn main() {
    let out = std::io::stdout();
    let mut out = BufWriter::new(out.lock());
    for _ in 0..100_000_000 {
        writeln!(&mut out, "Hello, world!").unwrap();
    }
}

```

```rust
$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:02 [611MiB/s] [<=>]

```

```rust
use std::io::{BufWriter, Write};

fn main() {
    let out = std::io::stdout();
    let mut out = BufWriter::new(out.lock());
    for _ in 0..100_000_000 {
        out.write_all("Hello, world!\n".as_bytes()).unwrap();
    }
}

```

```rust
$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:00 [1.58GiB/s] [<=>]

```

LTO does save things a little because the call to write\_fmt can be further inlined and optimized, but just calling write\_all is still faster:

```rust
$ cat >> Cargo.toml <<EOF
[profile.release]
lto = true
EOF
$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:01 [1.12GiB/s] [<=>]

```

LTO makes the write\_all version faster too:

```rust
$ cargo run -q --release | pv > /dev/null
1.30GiB 0:00:00 [1.89GiB/s] [<=>]

```

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [September 4, 2020, 12:58am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/2 "2020-09-04T00:58:06Z")

</div>

I don't think this is possible. `fmt::Write` doesn't have `write_all()`, and formatting macros need to support `fmt::Write` as well as `io::Write`.

---

<div class="post-metadata">

**Author:** ![glandium](https://avatars.discourse-cdn.com/v4/letter/g/3d9bf3/32.png) [@glandium](https://internals.rust-lang.org/u/glandium)\
**Post date:** [September 4, 2020, 1:00am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/3 "2020-09-04T01:00:31Z")

</div>

Oh right, there's that... but `io::Write` could have `write_str`.

---

<div class="post-metadata">

**Author:** ![glandium](https://avatars.discourse-cdn.com/v4/letter/g/3d9bf3/32.png) [@glandium](https://internals.rust-lang.org/u/glandium)\
**Post date:** [September 4, 2020, 1:02am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/4 "2020-09-04T01:02:54Z")

</div>

This makes me wonder why there isn't a `impl fmt::Write for W where W: io::Write`... maybe because that would make `write_fmt` ambiguous?

---

<div class="post-metadata">

**Author:** ![cuviper](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cuviper/32/1897_2.png) [@cuviper](https://internals.rust-lang.org/u/cuviper)\
**Post date:** [September 4, 2020, 2:56am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/5 "2020-09-04T02:56:14Z")

</div>

A blanket implementation of a trait has to be in the same crate as its definition, but these are `core::fmt::Write` vs. `std::io::Write`.

---

<div class="post-metadata">

**Author:** ![kornel](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kornel/32/2711_2.png) [@kornel](https://internals.rust-lang.org/u/kornel)\
**Post date:** [September 5, 2020, 10:10am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/6 "2020-09-05T10:10:35Z")

</div>

I'd love `write!` and `format!` to be smarter.

However, last time this was discussed (sorry, can't find thread), the answer was that these are just macros, and don't have type information. They could be made magically type-aware in the compiler, but that would make them different from all other macros.

---

<div class="post-metadata">

**Author:** ![Tom-Phinney](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tom-phinney/32/3299_2.png) [@Tom-Phinney](https://internals.rust-lang.org/u/Tom-Phinney)\
**Post date:** [September 5, 2020, 12:52pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/7 "2020-09-05T12:52:25Z")

</div>

I'd prefer that they stay as pseudo-macros, even to the point that it be possible to publish proc\_macro equivalents (now that proc\_macros are stabilized). Thus this is a plea for less magic and more regularity in Rust, not the opposite.

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [September 5, 2020, 2:08pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/8 "2020-09-05T14:08:54Z")

</div>

I know last time it came up, I considered the idea of a community group to work on proc macros that were significantly smarter. Unfortunately classes started this past week, so I don't have nearly as much time any more. However, I'd still be down for overseeing such a group if desired.

IIRC what came up last time was even things like `format!("{}{}", x, y)`, which could be slightly optimized to use the known lengths of x and y.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [September 5, 2020, 2:56pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/9 "2020-09-05T14:56:08Z")

</div>

As far as I recall, the formatting machinery included in std works, but basically nobody actually understands how it works entirely. Last I saw consensus was roughly that it'd be simpler to design a new solution (that fits the existing stable API) than to try to incrementally get the existing solution to the desired better end state (at least a portion of which would be allowing switching between fully-runtime-optimized or size-consious approaches.

(Also happy cake day @jhpratt)

---

<div class="post-metadata">

**Author:** ![tspiteri](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tspiteri/32/7283_2.png) [@tspiteri](https://internals.rust-lang.org/u/tspiteri)\
**Post date:** [September 5, 2020, 3:44pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/10 "2020-09-05T15:44:40Z")

</div>

> [@Tom-Phinney](#):
>
> Thus this is a plea for less magic and more regularity in Rust, not the opposite.

I'm not sure I see the value in this. The standard library is allowed to do things that cannot be done outside the standard library. And in the original topic suggestion, the interface itself is unchanged. I do not agree with the reasoning "An external crate cannot be optimized this way, so we should not optimize the standard library this way." Of course there could be other reasons not to do this, for example the optimization's implementation might be too hairy and not worth the development/maintenance cost. But if something in the standard library can be optimized with internal magic and the cost/benefit ratio is in favour of the optimization, I think it should be.

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [September 5, 2020, 3:57pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/11 "2020-09-05T15:57:09Z")

</div>

Yeah, it may have been that. Or both, I don't particularly remember that thread well. I just know there were some shortcomings that people had an interest in fixing.

---

<div class="post-metadata">

**Author:** ![Tom-Phinney](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tom-phinney/32/3299_2.png) [@Tom-Phinney](https://internals.rust-lang.org/u/Tom-Phinney)\
**Post date:** [September 5, 2020, 3:58pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/12 "2020-09-05T15:58:21Z")

</div>

Philosophically I agree on the optimization aspect of specialized compiler support, but not on the aspect that the compiler _magically_ implements things in a pseudo-macro that are impossible for others to implement in a real macro. For example, if the compiler uses type knowledge derived from post-macro-expansion type inference in its implementation, then I would prefer to see Rust extended to permit such capabilities added in a new form of later-compiler-phase-executing macros that have access to that same information.

`rustc`'s unstable plugin capabilities do provide means to achieve this, and more, but those plugins are for a specific implementation of the Rust compiler, so are unlikely to work with alternate-source compilers that will come into existence as Rust matures. In fact existing plugins are `rustc`-version specific, so sometimes require updating when new `rustc` versions are released.

---

<div class="post-metadata">

**Author:** ![pcpthm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/pcpthm/32/4952_2.png) [@pcpthm](https://internals.rust-lang.org/u/pcpthm)\
**Post date:** [September 6, 2020, 9:05pm UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/13 "2020-09-06T21:05:31Z")

</div>

This kind of ad-hoc polynomrphism can be done using the [autoref-specialization](https://github.com/dtolnay/case-studies/tree/master/autoref-specialization) trick without compiler magic: [Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=06f24afa82f900f3a8698e91d39d8d60)

```rust
struct IoWriteKind;
impl IoWriteKind {
    fn write_str<W: ?Sized + std::io::Write>(self, w: &mut W, s: &str) -> std::io::Result<()> {
        w.write_all(s.as_bytes())
    }
}

struct FmtWriteKind;
impl FmtWriteKind {
    fn write_str<W: ?Sized + std::fmt::Write>(self, w: &mut W, s: &str) -> std::fmt::Result {
        w.write_str(s)
    }
}

trait IoWrite {
    fn write_kind(&self) -> IoWriteKind { IoWriteKind }
}

trait FmtWrite {
    fn write_kind(&self) -> FmtWriteKind { FmtWriteKind }
}

impl<T: ?Sized + std::io::Write> IoWrite for &T {}
impl<T: ?Sized + std::fmt::Write> FmtWrite for T {}

macro_rules! my_write {
    ($out:expr, $str:literal) => {
        (&$out).write_kind().write_str($out, $str)
    }
}

fn main() {
    {
        let mut out = std::io::stdout();
        my_write!(&mut out, "Hello, world!").unwrap();
    }
    {
        let mut out = String::new();
        my_write!(&mut out, "Hello, world!").unwrap();
        assert_eq!(out, "Hello, world!");
    }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [December 6, 2020, 11:28am UTC](https://internals.rust-lang.org/t/should-writeln-expand-to-write-all-when-it-can/12997/15 "2020-12-06T11:28:41Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.