Idea: universal pipelining (a.k.a. making @await generic)

In “Await Syntax Discussion Summary” it was proposed to allow keywords to be used after ., for example:

await future
match foo { ... }
// will be equivalent to
future.await
foo.match {...}

Personally I am not fond of using . in such way, partially because it’s too similar to magic attributes and I think will be quite confusing for newcommers. But the idea on another hand is quite appealing. So how about introducing sigil for pipelining which (eventually) will work with keywords (accepting one expresion), functions and macros? Unfamiliar sigil will cause unfamiliar with the feature to learn about it, thus making it less confusing.

Let’s use @ for examples (open to bikeshedding):

let data = await connection.get();
data
    .unwrap()
    .iter()
    .filter(|v| { ... })
    .nth(3)
    @match { ... };
    .is_none()
    @if { ... }
let data = connection
    .get_header()
    @await?
    .get_body()?
    @await
    .unwrap();

// assuming generators will have resume with arguments
let val = value
    @yield
    .process()
    @yield;
// equivalent to
let t = yield value;
let value = yield t.process();
foo(24, bar(baz(data), 42));
// equivalent to
let res = data@baz@bar(42)@|x| foo(24, x);
// this functionality will effectively give us postfix macros
"Result: {}"@format!(res);
res@assert_ne!(None);
res
    @refine
    @dbg!
    @Into::into
    @check?
    @db.save? // `db` is a value which has `save` method
    .stats()
    @return;

Previously several pipilining proposals were posted (1, 2), but they haven’t found much traction. But maybe such generalization will be more successful?

UPD: Translation of real-life code taken from await-syntax:

29 Likes

Seems potentially interesting. Allowing some.long.list.of.stuff.for x { let y = foo(x); use(y) } could be nice for iterator chains sometimes.

2 Likes

I am not sure if it should work with for, we already have for_each which pretty much covers this use-case. Well, the only difference is that you have to be explicit with into_iter()/iter() in some cases.

Well, for_each doesn't work if you want to break.

(Yes, we have try_for_each for that, but that's not as obvious.)

6 Likes

This does seem very interesting and it would sidestep the debate entirely. We would just provide the prefix await syntax and allow you to do postfix with the @ sigil. I love this. Looking at the more involved parts, the @ symbol looks too noisy, is there another sigil we could use? (mabye ~ since it isn’t taken yet or we could do another arrow like ~>)

@newpavlov I would find it surprising if this worked with match and if but not for.

5 Likes

I don’t have strong feelings about allowing or disallowing postfix for, it’s just that items@for x {..} looks a bit strange and can not be read “naturally”, one reason for it is that in is not part of the expression anymore. On another side postfix for will work nicely with potential generator integration. Same can be said about if let. Either way list of allowed keywords can be extended at any time later.

As for sigil choice, I think we should choose from one-symbol sigils, so viable options are: @, #, ~. Exclamation mark probably should be excluded, as it will make a bit harder to parse code.

2 Likes

My knee-jerk reaction is also to support this "universal postfix syntax" with a special sigil, but I would like to have more time to think about the implications and look at some real code examples.

But if bikeshedding which sigil to use is on-topic, then I'd like to advocate for |, since we're already calling this "pipelining". (I haven't seen that suggested, but I may have missed it.) I don't believe it would be likely to be confused with the closure syntax.

1 Like

I think it would be valuable to add a version of this as a branch to the await-syntax repo.

2 Likes

But it would be ambiguous with regards to bitwise or, which also uses infix | in expression position.

I have a feeling that this will spin up yet another debate about the syntax, in which case it doesn't really improve the "sidesteps the issue" aspect of async-await syntax.

3 Likes

Here is my attempt to do translation. Some observations:

  • As expected fut@await? pattern is quite common, without await? fut sugar postfix style will be almost always preferred, as an alternative is much clunkier: (await fut)?. There is a nice symmetry between await? fut and fut@await? which could make the first variant easier to understand intuitively (even though the second variant is a combination of orthogonal features).
  • Another common pattern is fut@await.context("..")?. I think postfix variant will be always used in situations like this.
  • User has a freedom to choose the style of awaits, especially if await? sugar will be added. For example this part:
let st = get_client_sme(wlan_svc, iface_id)@await?
    .status()@await?;
// alternative formatting
let st = get_client_sme(wlan_svc, iface_id)
    @await?
    .status()
    @await?;

Can be written as:

let sme = await? get_client_sme(wlan_svc, iface_id);
let st = await? sme.status();

UPD: Here is a variant with await? sugar: gist.

1 Like

Inspired by elm, this feels like |> (right triangle/pipe into). But I suppose it would require an edition change to parse those as a single symbol instead of two? But perhaps \> might work since \ is a bit more special, I think?

1 Like

I’m not sure parsing |> would be a breaking change since I can’t find any code where |> would be a valid sequence today.

1 Like

They are valid within macros (e.g. they would match a pair of sequential tts instead of just a single one) and are emanated as separate tokens by the parser rather than a single one so that |> is indistinguishable from | >. Changing that would be technically a backward-compatibility hazard with macros AFAICS. Not sure how serious, though; would require at least a crater run to check.

But perhaps you’re suggesting that this token sequence could still be used for this purpose? Someone who knows the compiler better than me might be able to comment on that.

3 Likes

Ok I did not realized tt on macro_rules were grouping operator sigils. It’s strange because TokenStream on procedural macro keep them distinct.

I think that this is a very good solution and even a more general one which is always good. I would be in favour of really any sigil in this case as long as it was one character but I would vote for @.

As for semantic of this I would think that for the expression expr_1 @ keyword keyword_stuff should be equivalent in all cases too keyword keyword_stuff expr_1

Examples:

  • foo @ for x in { ... } is equivalent to for x in foo { ... }.
  • foo @ if let Some(n) = { ... } is equivalent to if let Some(n) = foo { ... }
1 Like

This feature definitely looks interesting and useful, but the @ sigil does add a lot of noise. Personally I’m not against multiple-character symbols, as the proposed |>, even if there’s potential macro breakage

5 Likes

I like the idea of a pipeline operator. I’m not so sure if it should work with if/match/… . Bikeshedding the syntax, I like the one of Ocaml and Elm: |>, which has some kind of visual indication of a pipeline.

4 Likes

I think postfix macro would be enough. It seems possible to write foo.match! { ... }, foo.for! { ... } etc. macros. And it doesn’t require @ which simply does not look good.

4 Likes

What does “adding a lot of noise” mean? A lot of people have been using that phrase recently. To me it looks like “I don’t like the symbol” and that is it, no further thought, and no further objections beyond that. @ has a history of being a “special character” in a large amount of languages (matlab, python, java, ruby, CMake script, php, Ocaml etc…), using it in pipe-lining here wouldn’t be without precedent (every single on of those languages uses the @ symbol in a different way). “It looks weird” is not a critique any more than thinking a stop light should be red yellow purple because you don’t like green.

Here’s a proper critique of the @ symbol. The @ symbol may conflict with pattern binding which uses the same symbol in potentially similar parsing scenarios. That’s a pretty big deal and does not rely on whether or not I think this symbol is “icky”.

|> might look nice, but as others have already pointed out in this very thread, it also has potential parsing ambiguities, among other potential parsing issues like re-using other symbols used for different purposes. Of course if one could prove that these are not issues, then that would support using this symbol.

We need to make our arguments more than “today I don’t like this symbol” if we want to have productive discussions. Visual signaling of what symbols do is certainly something that can be used as justification for using one symbol, precedence is also a justification, emphasizing/not emphasizing something is also a justification, but only for deciding to/not to use a special symbol at all. “it looks weird” or “it looks noisy” is not a justification.

What’s more is that simply stating “I like option X, I don’t care about issue Y” is noise in a thread, and should be avoided. Leave that for polls. Instead state “I like option X, issues A B and C aren’t a big deal because of Q, R and because of what S said here[link], Issue Y is not a big deal because of E”. That moves discussion forward. These threads already get big enough as it is.

6 Likes

It means the symbol makes the code more information dense, more intimidating to newcomers, and require more concentration to mentally parse. In effect, it makes code less explanatory and more like jargon, which is unfriendly to people who do not specialize in it or people who switch contexts regularly.

4 Likes