Especially with discussion around a potentially adding a pipe-forward operator, I’d like to present an idea I had for partial function application.
Today, let’s say I have some monster of a function
fn foo(a: u32, b: u32, c: u32, d: u32, e: u32);
And I want to turn it into a FnOnce(u32, u32) by supplying it some values ahead of time. Today, I can write that as:
consume(|b, d| foo(5, b, 15, d, 25))
But the partially applied idea is to allow _ as a stand in for deferred argument provision, so this could instead be written as:
consume(foo(5, _, 15, _, 25))
These two would be equivalent.
So, what pros/cons do you see in this design? I’m not really sure how useful this would practically be, but partial application is often brought up when piping data through more functional pipelines.
Rather than building this into the language, I’d prefer to see this implemented on top of parameter-pack-style variable arguments. eddyb has some plans for how parameter packs would work. With that, it’d be possible to add a method on Fn (and FnOnce and FnMut) that took the same type as its first argument (or Nth argument) and returned a function of the remaining arguments. Wrap that in some macros and you could easily implement this without any special language support.
It seems to me that consume(|b, d| foo(5, b, 15, d, 25)) is clean enough. I don’t think that Rust need a new syntax for this. It is neither very common nor painful to write/read with the current syntax.
Seems to me that this would not add much value to Rust.
In my opinion, Rust needs more work on fundamental features (GATs, async/await, const fns etc) and not expand its surface syntax for things that it can already do.
The difference between them is that the first category actually makes Rust capable of things it cannot do at the moment, while the latter category just expands the complexity of the language for very questionable return.
I'd say that partial application or "auto currying" (not necessarily this particular proposal..) is immensely good to have for ergonomics, in my experience. This is partly why you can't beat a language like Haskell in expressiveness.
It would be quite nice to be able to take let lam = |a, b, c| expr; and do lam(1) and get back another lambda until it is fully applied with lam(1)(2)(3). This is particularly useful for iterator chains.
I know, but it's complex work, and thus (at least from my perspective) slow going
I agree that partial application works well in Haskell, but I think it's useful to keep in mind that much Haskell was designed around at leas the assumption of that being available. In Rust at least so far that is of course not the case, thus I would expect that making partial application really pay for itself would require additional changes e.g. making it easier to assign closures returned by a partial application to fields, ideally unboxed if at all possible.
Also, and this is probably just a lack of imagination on my part, I fail to see how exactly partial application could be useful in Rust? As in, where in the usual going on in Rust code, would one partially apply a function and why?
Sure; Hopefully we can still make it work well after the fact in Rust. But perhaps we can't. Time will tell ^,-
Naturally. Partial application could just be sugar for creating another closure. That way you can store the returned closures in fields and pass them around.
fn foo(x: u8, y: u8, z: u8) -> u8 { x + y + z }
let a = foo(0); // { let arg = 0; |v0, v1| foo(arg, v0, v1); }
let b = a(1); // { let arg = 1; |v0| a(arg, v0); }
let c = b(2); // { let arg = 2; b(2); }
fn foo(x: u8, y: u8, z: u8) -> u8 { x + y + z }
let a = move foo(0); // { let arg = 0; move |v0, v1| foo(arg, v0, v1); }
let b = move a(1); // { let arg = 1; move |v0| a(arg, v0); }
let c = b(2); // { let arg = 2; b(2); }
I haven't studied the implications of this in depth so there could be technical problems with this approach.
A trivial example showing why partial application is useful:
This example is of course very small, but dealing with iterators in general would be helped.
This sort of thing of having to introduce a lambda instead of partial application bothers me a lot at the moment when writing Rust.
I see your point. At the same time I have to wonder if it isn’t actually a good thing to see the “closure noise” as it potentially aids comprehensibility, in the sense that it makes the data flow through the closure and foo() call explicit.
It’s not something I have an answer for atm, I don’t feel like I have enough data on complex examples to see how it would play out there.
And this “closure noise” is actually the reason for this idea using _ as a parameter standin. There’s no suprise-capable calling of a function with not enough arguments, as the positions are filled. And because they’re filled with a placeholder, it can be understood that it’s a delayed evaluation of the partially applied function.
What I’m wondering is whether it might actually be a good thing to have that noise. The _, while succinct, obscures what those arguments are supposed to be, or their types for that matter. This definitely makes the code shorter, but does it make it more comprehensible, or less, or does it perhaps make no difference at all?
Comprehensibility is a factor of the reader more than anything and depends on what you are used to. For example, I find pointfree composition of functions much more readable than introducing temporary bindings, lambdas (which just introduce temporary bindings…), etc.
areaOfBlackVolvos cars =
fmap area $
filter (\car -> maker car == "Volvo") $
filter (\car -> color car == Black) $
cars
Naming is a hard problem in software engineering, I find, so one shouldn’t try to force the developer to name up names for temporaries and instead allow them to focus their energy on naming what matters well. My experience from Rust code is that lambdas introduced due to lack of partial application usually contain poorly named binders. Often they are one-letter binders, which don’t help in understanding.
I totally get the argument, that this does not add anything that is not possible today and the closure syntax is already quite ergonomic… but i‘d love to see something like this in the language.
Most if my closures in iterator chains are filled with single character names that are used directly. I want these parameters to be anonymous for the same reason i want the closure to be unnamed in that situation.
This is actually something I’ve wanted for a while, since I came to Rust from Scala.
One particularly annoying pattern I’ve come across is wanting to turn an impl Iterator<&T> into an impl Iterator<T> where T: Copy. This can be accomplished with a very simple closure: iter.map(|x| *x), but I can’t help but want to reach for iter.map(*_), as I might have written in Scala; having to pick a name for the variable (and write it twice) feels like an unnecessary papercut. I often want to reach for _.foo(..) where I would use |x| x.foo(..) as well.
At the same time though, this syntax hides that a closure is being created, which feels like going against Rust’s promise of making such things very explicit. Implicit currying is even worse (which even Scala doesn’t allow):
fn foo(x: i32, y: i32, z: i32) -> i32 {..}
let bar = foo(1, 2, 3); // normal function call
let baz = foo(1, 2); // actually a closure definition!
For someone coming from a language with function overloading, this behavior is surprising (and for which Haskell is not prior art; Haskell does not group function parameters in parentheses).
What I would like to see is something like iter.map(|| *_). The bars indicate to the read the creation of a closure, inside of which _ is a stand-in for a variable. This presents a few questions:
|| expr is no longer always a nilary closure, dependent on the presence of underscores in expr. Should
we use some other syntax, like |..| *_? (This is a strawman; this much punctuation is pretty ugly and hard-to-read.)
What does |x| x + _ mean? Does it desugar to |x, $0| x + $0? Should we allow it at all?
What does || _ + _ desugar to? I agree with above discussion that it should be |$0, $1| $0 + $1, but I’d like to hear an argument for not having a slightly-more-explicit syntax as a shorthand for |x| x + x or |x, y| y + x.
For example, given iter: impl Iterator<(i32, i32)>, I’d like to write iter.map(|| _.0 + _.1), orsomething along those lines.
For some prior art on the third bullet, in Mathematica we have the following behavior:
I’ll note that Mathematica makes a number of questionable syntax choices, not the least of which is lacking a lambda syntax (other than Function[]) that doesn’t involve numbered parameters.
But Rust is not a language with function overloading.
Haskell does group function parameters in parenthesis if you pass them as a tuple.
foo :: Num t => (t, t) -> t
foo (a, b) = a * b
f12 = foo(1, 2)
This is perfectly valid Haskell. However, you can't partially apply as foo(1) in this case. So you are right that Haskell isn't prior art; however, in Haskell, you typically write this as:
foo :: Num t => t -> t -> t
foo a b = a * b
f12 = foo 1 2
In Rust, what you're really doing is passing a tuple:
fn foo (x: i32, y: i32, z: i32) -> i32 { .. }
I can see how this implicit currying would be problematic from the perspective of a haskeller.
But it is quite useful when we take the ecosystem into account.
I wonder if we could do something like:
fn foo (x: i32) -> (y: i32) -> (z: i32) -> i32 {
x + y + z
}
This would not be sugar but explicit currying and different in the type system.
Sure, I think my wording was poor here. The point I meant to make was that, AFAIK, there is no place where it is possible to create a closure without a double bar, and I think that sort of visual signaling is useful. I spend a lot more time reading others' code than writing my own, and being able to see at the call site that a closure, and not a value, is being passed is valuable for readability. For comparison, Scala's def foo(f: => A) (by-name parameters) creates lambdas out of bare expressions, which are convenient when writing code but make it frustrating to determine if a function call is secretly constructing a lambda, since I might need to jump to a different file.
I think one can argue that this sort of sugar is comparatively harmless, like auto-deref, but for now I remain skeptical.
I don't know that I suggested otherwise? I assumed your proposal was like Scala's foo(1) _ syntax, rather than Haskell's bona-fide curried functions.
No, but function overloading is a feature present in many popular languages (e.g. Java, C++) that is orthogonal to currying. This syntax would probably leave a lot of new Rust users scratching their heads. I don't think an argument can be made for this like was made for using try blocks; at least try blocks involve code that needs error-handling.
I think comparing Haskell and and Rust syntax is just going to lead us astray. For example, in Haskell, I can write the following, since every function in Haskell takes exactly one parameter (modulo sugar to make it look like functions can take multiple parameters).
This does not work in Rust, since functions can take more than one parameter:
fn foo<T>(a: T, _: T) -> T { a }
let t12 = (1, 2);
let f12 = foo(t12); // error[E0061]: this function takes 2 parameters but 1 parameter was supplied
// this is the only correct alternative
let f12 = foo(1, 2);
Not sure why that is valuable; after all, a closure is a value. If the function foo was explicitly written as:
fn foo(x: u8) -> impl Fn(u8) -> u8 {
move |y| x + y
}
... you would not be able to tell from foo(x) whether or not a lambda or a non-lambda is returned..
However, the language is currently making currying quite cumbersome even with -> impl Trait; If you write foo this way then you can't write foo(1, 2) anymore.
Auto-deref is a great example of such implicitness working fine. I don't think implicit currying is any more magical than that, or more problematic.
I was clarifying for the audience
I'm not particularly familiar with Scala; got a link to the documentation?
Fair enough; I think it is fairly easy to teach this tho and can be done early in the teaching material.
This is true. But we could also do that;
fn foo (x: i32) -> (y: i32) -> (z: i32) -> i32 { x + y + z }
Here, foo is a proper curried function. It might not be particularly useful to do this distinction and instead just allow the implicit currying because it would work well with the current ecosystem and solves the problem equally well while at the same time being a much smaller change.
Closures are values, but that's missing the forest for the trees. Suppose I'm reading another engineer's code and I want figure out what their intent was:
// here, it's immediately clear that a closure is capturing
// behavior to be executed elsewhere
foo.do_something(|x| bar(1, y, x));
// it's not immediately clear if the value being passed is
// a scalar (whatever `bar` returns) or a closure;
// in fact, I'd assume that the return value of `bar` is
// what's being passed
foo.do_something(bar(1, y));
Of course, this is a strawman, and it ultimately comes down to taste. (Warning, personal opinion incoming) Rust has just the right amount of local inference that I can look at a piece of code and guess what it's doing without looking up every function declared in a different file.
I think the jury's out on that. Our opinions on the matter are clear, but let's wait and see what other folks think.
This SO answer gives a neat explanation of it.f(x) _ desugars to y => f(x)(y), as in your proposal. AFAIK the underscore is there to disambiguate type inference (since Scala has overloading), and not to make it clear that what comes out is an anonymous function. This is opposed to Haskell's currying, where functions truly only take one parameter and always return functions, so that f x y z is really (((f x) y) z).
Too little too late, if you ask me. Haskell as designed with currying from the onset, so all of the standard library functions have parameters in "the most reasonable order" for currying purposes, e.g. map :: (a -> b) -> [a] -> [b]. Not so for Rust; for example, Iterator::map is fn (impl Iterator<Item = T>, impl FnOnce(T) -> U) -> impl Iterator<Item = U>, which limits potential usefulness of currying. Being able to write || _.map(f), with or without the bars, would achieve a closer purpose.
Again, I don't think we're going to convince each other; I'd like to see what other people think of our respective proposals.
Don't say never. I was somewhat neutral on adding (x: i32) -> (y: i32) -> ...
You've at least convinced me that this is a bad idea. Your note about designing the standard library (and other notable packages/crates) with currying in mind is spot on and very true.
