Idea: Macro for constructing Strings without format!

@steffahn Thanks, I suppose I'm interested in wherever the alloc crate is discussed.

This functionality seems in line with other macros that are in std is why i'm even bringing it up here at all as a proposal for alloc.

I see. I'm not entirely sure what the criteria for inclusion into the standard library are or which macros you are referring to. As far as I know, the format! macro uses special compiler support, and thus needs to live in the standard library.

I'm not very well versed in the standard library regarding string. I just noticed that there is a concat macro for combining literals in the standard library, so I suppose your proposal might make some sense. (Edit: Just noticing that that one is built into the compiler as well)

Another idea could be to somehow optimize format to have less code-size impact for uses that are simply concatenating stuff.

1 Like
1 Like

I'm wondering how well the concat method on an array literal gets optimized, in comparison. I'll test that when I'm home.

1 Like

There is also the concat! macro which you can use if you are only dealing with literals.

1 Like

FWIW, there's never been anything resembling official criteria. In practice it's more of a "can you convince people that there's a significant benefit to putting it in std instead of a crates.io crate?", and 99% of the "X should be in std" threads that pop up around here never come close to clearing that bar.

As far as extrapolating some unofficial but nonetheless informative de facto criteria based on what's actually gotten into std historically, I don't think much has changed since I wrote this:

Typical compelling arguments for putting things in the Rust standard library include "vocabulary types" (e.g. the whole Rust ecosystem needs to agree on a single str type just so everyone's code can talk to each other), code that needs to interact with the compiler in some magical way, or tricky unsafe operations (because we trust std verification more than 3rd party crate verification, though even that could potentially change in the future), or platform abstractions that help keep a lot of Rust code portable-by-default (though that strategy didn't scale as well as we hoped, so that might change too). All of these are pretty fuzzy and case-by-case criteria, not remotely official in any way. In fact, in the specific case of an HTTP server, there is an obvious vocabulary types problem, but the ecosystem ended up agreeing on a 3rd party crate for that and AFAIK no one's ever asked for it to be moved into std.

The main thing I left out there which is probably noteworthy in this discussion is "tiny things that nearly everyone would find useful but would never import a crate for", which is very rarely a successful argument, but the poster child for succeeding that way is the dbg! macro. IMO that's the most relevant precedent to compare against, at least for the "is this worth putting in core/std?" question.

5 Likes

I’m having a hard time understanding how optimal the generated code is when doing ["Hello", name, "!"].concat() (optimized asm). Apparently the loop is unrolled, but the array is still fully created on the stack and some parts access it while other parts of the code just directly re-create its contents. All in all it doesn’t look too bad, and I prefer this syntax (in terms of code readability) over a new variadic macro.

1 Like

Really great point! Thanks for pointing out this method.

First, concatenating literals is a very different use case from concatenating strings.

The advantage of format! is that

  1. it has several options how a value should be formatted
  2. it can format almost anything, including strings, numbers, booleans, arrays, tuples, and many enums and structs
  3. It usually doesn't need extra allocations

While 1. often isn't needed, formatting other types than strings (especially numbers) is quite common.

You can also use the μfmt crate as a lightweight alternative to format!.

3 Likes

Seems like quite a useful crate. It does make me wonder what it had to give up in order to achieve the size and speed gains. If the answer is "nothing", then it seems like this crate should be promoted over the code in stdlib, similarly to plenty of extant examples like the new HashMap implementation, crossbeam's MPMC channels being strictly superior over std::mpsc etc.

1 Like

One issue with promoting it as a direct replacement for the stdlib code is the non-features:

Non-features

These are out of scope

  • Padding, alignment and other formatting options
  • Formatting floating point numbers

Though perhaps another library could wrap μfmt to provide this functionality.

1 Like

I thought most of those formatting options required allocating a string, so you simply can't implement them in no-std environments anyway?

There's definitely a desire to replace much of the std formatting machinery with a new design that's more better. I think nobody actually understands how the current formatting plumbing actually works :sweat_smile:

The problem is that replacing it is a lot of hard work, because it does a lot more than you'd like to think, so a lot of the complexity is intrinsic.

7 Likes

As far as I understand, currently format always builds Arguments object, with dynamic dispatch, etc.

I've always wondered why there's no optimizer for format!. It's in std, and already a compiler-built-in magic, so the magic could go further and skip format_args!() machinery entirely whenever possible.

It'd be awesome if format!("{}{}",a,b) to compiled down to

let mut res = String::with_capacity(a.len+b.len()); 
res.push_str(a); res.push_str(b)

which in turn optimzied down to a couple of memcpys.

12 Likes

I'd personally be interested in attempting to write a better format! macro using proc macros. Not by myself, but with others, as it certainly wouldn't be easy.

If anyone else would be interested in this, let me know. Maybe we can organize something!

Edit: With regard to contacting me, send me a PM. I'll probably set up a repo on GitHub if there's a few others interested.

2 Likes

I think there's still desire for format! to continue to pretend that it's "just another proc macro", even if it is implemented directly in the compiler. That would mean it doesn't have type information available.

1 Like

For those following along at home, why does format! need compiler magic?

std::fmt::Arguments can only be created by the compiler, as its fields are private. However, every part of the chain above that can absolutely be done in a proc macro.

Although. I don't see why it shouldn't be possible to have an unsafe public API for the construction of a std::fmt::Arguments. I'm not familiar with its internals, though.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.