Named & Default Arguments - A Review, Proposal and Macro Implementation

language design
1

I’ve written a review of previous RFCs concerning named & default arguments in Rust. It covers a lot of the considerations of such proposals, reviews the history and then puts forth a more stepwise path to getting named and default parameters into Rust.

I’ve also written a nightly only proc-macro that lets you use write a function, and call it with positional arguments, or calling it in macro form, call it with named arguments. It’s quite limited in scope but I hope to grow it and polish it as a form of exploring this problem space.

Link to Review

Link to Macro Implementation Readme

I’m interested in any feedback you may have. I’d like to polish the review more, going more in depth into the alternatives. But I also just wanted to get this out there and get feedback.

16 Likes
2

Wow, that’s an epic summary! Thank you for putting all that together.

Also, I’m a huge fan of the “make it easier to use structs” approach to this. I especially like that it lets you build up arguments in a struct if you need to.

This example is already valid rust, so I don’t think we can do this:

fn the_answer() -> Demo {
    let t: Tupled = Tupled(1,2,3);
    let answer = t.0+t.1+t.2;
    {answer}
}

It’s even used in real life, for things mentioned in https://bluss.github.io/rust/fun/2015/10/11/stuff-the-identity-function-does/#rust-has-dedicated-syntax-for-this

One thought I had on syntax: what does Foo(a, b) mean today?

Did you say “tuple struct construction”? Because it’s not; it’s a call to fn Foo(x: i32, y: i32).

So if we’re good with bar(a, b) and Bar(a, b) both existing, it feels like we should also be fine with cat{x: a, y: b} and Cat{x: a, y: b}. And then it’s the same parse that already exists, just desugared (after name resolution) to a function call + struct construction.

That way it doesn’t need new Fn* traits, it works with existing methods that take a struct, etc.

2 Likes
3

Wow, nice summary! Thank you for your efforts.

For my part, I feel like trying to jump to a solution that uses structs rather than simply using Option for default arguments seems like it introduces unnecessary complexity. Since the proof-of-concept will rely on Macros, why not simply have the macro create a function that wraps all arguments with default parameters in Option and inserts the code inside the function definition to map the Option to the default value if it is None. Then on the calling side, just sub in Option<None> for all not provided parameters with the macro that does the calling side. This seems much more straightforward then using an intermediate struct.

EDIT: It seems like a good way to handle getting to Named Parameters in a step-wise fashion would be:

  1. Make the compiler able to elide Optional Parameters on Functions/Methods
  2. Add annotation to opt-out of Optional Parameter elision at the definition site - Also, implement annotation or syntax for API to define a default value beyond Option<T>/None and T:Default
  3. Make naming parameters at call site optional (either all named, or none named) in order
  4. Add annotation to opt-out of either named or non-named parameters at the definition site
  5. Allow eliding name of parameter in parameter calling convention if the variable being passed exactly matches the name of the elided parameter name (and the name is not already given on another parameter)
  6. Add an Opt-In at the definition site for allowing out-of-order named calling convention
  7. Add ability to overload function/method names that require named calling convention with multiple versions with different sets of named parameters

So, how would this look? First, an RFC and PR to allow #1. This would make any parameter that is Option<T> or T : Default permitted to be left off from the call site. This would be a backwards compatible, non-breaking change because it would not break in any way existing call sites or existing API’s (that weren’t already broken). Calls would be required to still include parameters in order of definition, but, Option<T> and T : Default parameters could be left out - the compiler would automagically insert None or T::Default() as appropriate for elided parameters at the call site.

Next, an RFC and PR for #2. This would allow the definition site to mark Option<T> and T : Default parameters as Non-Optional. This would be a breaking change to an API (only if the API chose to exercise this option), but, if done soon after #1, would not have a wide-ranging impact and could be done only on a case-by-case basis where the API deems it necessary. It seems like this is unlikely to happen very often as it would only enforce something for the purposes of explicitness not for correctness. Also, we could allow the API author to annotate specific parameters as having specific default values for any parameter in the function/method definition and have these parameters treated as default parameters as well. This should be 100% backwards compatible and require no API breakage.

Then, an RFC/PR could then enable #3 the ability to call any method/function with named or unnamed calling convention. No mixing of the two would be permitted. It is either call using all parameters (except elided optional parameters) with no names in definition order or calling all parameters (except elided optionals) with named arguments (still in definition order). This again, would be a 100% backwards compatible, non-breaking change.

Now, an RFC for #4, the ability for the definition site to opt-out of either Named or Unnamed calling convention. If no annotation is given, then, the method/function can be called using either calling convention. If the annotation is given, that either opts for forcing either named or unnamed convention, then, that selected convention would be required at call sites. This again could be a breaking change for a given API (only if the API chose to exercise this option), but, again, it seems unlikely that this option would need to be exercised often as it again would be about explicitness and enforcing a particular idiomatic calling style not correctness.

Now, an RFC/PR could be implemented for #5 to allow eliding the name of a parameter in the named parameter calling convention style if the name of the variable being passed exactly matches the name of the parameter and that parameter is not otherwise already named in the call. This would be a 100% backwards compatible change and would not break any existing API’s or call sites.

After that, add an RFC/PR for #6, the ability of the definition to opt-in to allowing out-of-order named calling convention. Without explicit opt-in, then in-order is enforced at the call site. With the opt-in the call site may call using named-parameters in an out-of-order fashion. This again is a 100% non-breaking backwards compatible change.

Finally, add and RFC/PR for #7, the ability to have multiple overloaded, named calling convention methods/functions, of the same name. This would allow method overloading based on the set of named parameters and would only be permitted for methods/functions that required named calling convention. In other words, the names of the parameters become part of the method/function name implicitly. This may or may not be a breaking change for API’s to add overloaded methods/functions depending on how this is specifically implemented. For example, if could be non-breaking if the first definition of a named-parameter only calling convention method did not make the implicit name of the method function depend on the names of the parameters but following definitions did. I really something like this could be useful/desirable and implemented in an API backwards compatible fashion.

I think a plan like the above breaks off the work into manageable chunks with no significant breaking changes anywhere in the process. It also leaves room for bike-shedding on specific syntax issues and permits opt-in/op-out where that seems prudent without introducing breaking-changes at the point where that functionality becomes available. In addition, each item stands on its own and we could stop anywhere in the process and still have the usefulness of the previous items without breaking anything or precluding future movement on the other items.

Thoughts?

BTW: I hope this isn’t taken as me attempting to hijack your discussion or ideas. I really like what you’ve summarized and am excited to see movement on this. I 100% would like to have named parameter calling convention and I’d be more interested in getting there than how we get there specifically, but, I felt that the above thoughts could provide some positive contribution to the discussion.

4

@gbutler High level comments about having another step-wise proposal:

I appreciate your comments. I think that there are two fundamentally different routes - I laid out a step-wise process for one, and you laid out one for the other. I’d call mine a sugary struct approach, I’m not sure what I’d name yours - maybe a native impl approach?

I think in order to get named & default arguments into Rust, we’ll have to examine in depth the pros & cons of both major paths, which includes a step-wise approach for both.

The major questions I have about the native impl approach is how it affects the type system. I’m assuming (but not entirely sure - I haven’t pondered it long enough & I’m not expert), that for this to work we have stuff the field names into the type system / Fn traits somehow. Or at the very least, we need to consider how this interplays with functions defined on traits (e.g. a trait implementation has to match the names used in the trait definition, and what about defaults), and how does it interact with closures. These seem like they could have reasonable answers to them, but I think to consider it as a path forward, those answers must be considered part of the cost.

What I love about the struct sugar approach is that we don’t even need to consider that question - we already have structs to encode name: value pairs in the type system. And with struct sugar any improvements to structs are trivially carried over to named & default arguments for functions. We get to use structs instead of modifying the type system.

Going back to your first point, I think yes, conceptually it seems a little heavy or awkward to use structs for default arguments. OTOH, from a language perspective it’s a lot lighter - no type system changes, just a couple of small modifications for structs. And if we have the proper sugar in place you don’t even have to think of it as structs - that’s just an implementation detail. This does of course hinge on any overhead incurred by struct construction / matching being optimized out - I’d think that’d happen, but it is a question that needs answering before formally proposing the struct sugar path.

I’ll need to think about your step-wise proposal. My initial thoughts is that it’s really heavy, and to avoid breaking changes #1 & #2 should be combined. And maybe make things opt-in so semantics don’t change for api authors. I think your proposal hints at more than just a divergence of how to implement things (struct sugar vs. not), but also in the semantics about how functions should be callable when they have named parameters.

I’ll need to chew on it a bit more to give a proper response to your specific proposal

2 Likes
5

Converting existing function definitions that use Option and impl Default types is deeply scary to me. I would prefer that optionals and default values start as opt in. I'm also not sure they are the right traits or types to use for optional or default parameters.

For instance, what parameters does this function require you supply? fn optionals(foo: Option<i32>, bar: NoImplDefault, baz: Option<u32>)

For Default there's a large gulf between "default value for a type" and default value in this function signature. I often want bools to be default true. Having fn takes_bool(default_true: bool) take and type check as takes_bool() with a default value of false seems like a tremendous footgun.

5 Likes
6

@scottmcm - That’s an interesting point about tuple struct construction. I think I saw it mentioned somewhere but didn’t have enough context at that point to stick. It would be a great parallel.

Regarding the quoted code - that’s valid syntax, but it would fail to compile because the types don’t match. Part of my proposal is that we type check blocks like {answer} to see if it should be treated as a one-field struct or as a block returning the value of answer. Granted, there are cases where this could be a breaking change - consider the situation where the return type is an impl trait, and both the struct and the single field inside it both implement the trait. So it’s definitely a speculative proposal, and the function call with braces is a good alternative (I think).

One thing I do like about the nested structs syntax (e.g. fn_name({})) is that if you really wanted to, you could use multiple structs. But that might not be good. I think my opinion is more about adding less special casing, and less about more flexibility :slight_smile:

1 Like
7

Allow using and declaring a struct inline inside a function declaration with the named keyword.

:bike::house: the struct keyword should be unambiguous in this position. In fact maybe the struct grammar could just be extended to support struct _ { /*..*/ } and when used in a function signature all of the field names have the same visibility as the function.

While type ascription RFC hasn't been merged and this is the most contentious part, I'd be curious about what the potential interaction of type ascription patterns in function signatures is with anonymous structs. Would we eventually be able to write:

fn with_optionals(required: &str, _ { foo: bool, bar: i32 })

Is that even desirable? It could be a more radical way to avoid the named keyword and introducing a new named parameter assignment operator (=> in your proposal).

8

There's a bunch of pain points I predict in this step particularly:

  • Right now, I can change my function argument names whenever I want. This freedom is immediately lost if rust starts allowing everything to be called with named parameters. (C# has this misfeature, and I find it very frustrating there when trying to refactor things.) So I think that it's critical that a function have chosen a nominal of some sort (perhaps annotations, perhaps a struct, ...) before it can be called in a nominal way.

  • What are the argument names when you call Fn* traits? What names do you use if an impl for a trait uses has different names than the trait did? What do you do if there isn't a name because the method declaration in the trait just used _ or because this particular implementation is currently ignoring it and thus uses _ or because the pattern isn't a simple binding, but something like fn foo([a, b, c]: [i32; 3])?

9

This kind of fallback tends to be particularly troublesome once the situations get more complex. Like does {{a}} mean S{a:{a}} or {S{a}}? The parser would have to generate some sort of "I'm not sure" AST that would get changed into the actual expressions after type checking, or something? It's pretty awkward.

Also, I think the parser would rather not need to use lookahead to decide what to do on seeing a {. (Though I wish bare braces would just work...)

:+1::+1::+1:

10

I find any “opt-out” strategy like @gbutler’s to be indefensible, especially when it changes the meaning of existing signatures, existing types (Option), and existing traits (Default).

By “changing the meaning of traits,” what I mean is, do you think f64 or Vec would impl Default if rust 1.0 had these semantics? I certainly would hope not!

I must thank @samsieber for this careful review and for making a proposal that takes into consideration all of these past discussions. Understanding the past is truly the only way to move forward and stop repeating history on such a hotly debated topic.

1 Like
11

I’d like to through in my idea in the mix: although the shortage of ideas is not a problem here, and I really haven’t thought about relative pros and cons, I’d rather tell it (and maybe someone runs with it), rather keep it secret. It might have been proposed before, but I haven’t seen it.

We can extend arity-based overloading of this RFC to handle arbitrary named arguments. Instead of dynamically wrapping kwargs into struct/options we do static dispatch based on call syntax.

A call like xs.sort(by = |x| x.abs()) is desugard to roughly xs.sort__by__(|x| x.abs()) where desugared function named is generated from the call expression syntax. Similarly, fn sort(by = _ : impl Fn(&T) -> R) is desugard to the same name fn sort__by__(by: impl Fn(&T) -> R). Desugard names are not available directly: if you want to make a closure out of a function with keyword arguments, you have to wrap in in lambda to have param = value syntax. If you want different or optional parameters (.sort(by=), .sort(cmp=)), you’ll have to define the function several times.

Nice things about this approach is that it’s zero overhead (there’s a separate overload for each combinations of parameters) and that it doesn’t really introduce new language features (no new closure types, no new type-inference rules (desugaring is purely syntactic)).

2 Likes
12

@matklad I think I understand what you’re saying - so generating one function impl per combination of named arguments from a single definition? I actually see that as being RFC 5 (after all the struct sugar stuff & inline struct declaration) if we decide that it should be a first class citizen. We could make it non-breaking by also generating a version that takes an inline struct instead of separate keywords.

13

I’ve also though a lot about a design for optional arguments, and I’m of the opinion that in an initial design the default value should be None, always. Allowing custom defaults leads to difficult questions about how to write defaults and what values can be defaults, which are better left for later extensions.

Since we’re doing proposal brainstorm, I’m going to throw mine out there with little detail, just focusing on the key features. You’d declare an argument as optional by prefixing it with a keyword such as opt, as in opt y: String. You can think of opt as a binding mode, because the value is passed in as a String but bound as Option<String>. For example:

fn foo(x: i32, opt y: String) {
    let z = y.unwrap_or("a default".to_owned());
}

An optional argument will be passed as None if it’s ommited from the call site, values for optional arguments must be passed by name, as in:

foo(0, opt y = "value");

The keyword opt is used as a separator between the mandatory and optional arguments, this is a grammatical necessity to avoid ambiguities, because y is not just a name but can be a pattern, such as in:

let maybe_a_string = cli_args.value_of("arg");
foo(0, opt Some(y) = maybe_a_string);

If the pattern matches, then the matched value for y will be passed in. If the pattern is refuted, then y is passed in as None. This would be the killer feature of this proposal because it allows the caller to ergonomically pass down uncertainty about the presence of a value. This is much more expressive than a builder, with a builder this sucks as you have to write the boilerplate:

if let Some(y) = cli_args.value_of("arg") {
  Some(y) => builder.set_y(y)
}

So it’s at least 3 lines of code for each parameter.

14

@ExpHP - I think the opt-in nature of @gbutler makes his suggested path less plausible, but it definitely could be altered to be opt in instead (the pub keyword suggestion by @Azerupi in the most recent internals thread (2016 I think) is a really elegant suggestion), while keeping the general direction (ingrained compile support instead of struct sugar) of that plan the same.

I want to mention that because I think that the two things are different concerns at play here, namely the opt-in/opt-out debate and the ingrained vs sugar debate. And I think they can mostly be decided separately.

Personally, I don’t think anything that requires opt-out for named arguments would ever make it as an RFC, though I mention that (though a little more indirectly) in my guide.

15

@leodasvacas I think what you’re suggesting is still strictly less powerful than the builder pattern - you can do more validation inside the builder, even preseeding the builder with the correct defaults. When I saw power, I mean capability, not noise to signal ratio (e.g. boiler plate).

It definitely is shorter to write & use - and that’s the trade off - more concise, but not quite as powerful (capable).

1 Like
16

@scottmcm Dang, I forgot about that RFC. I think I came across it while doing research on this topic. I’ll have to go add it into the notes when I get time :slight_smile:

17

I don’t think it makes sense to allow omitting any function argument whose type impls Default. For example, integers impl Default, so that would mean that all integer arguments to functions become optional with a default of 0. In the majority of cases, that would make no sense.

10 Likes
18

Perhaps it might be a better idea to turn the need for specification in optional arguments around. Instead of being omitted entirely, placeholders are put in for arguments for which the default value should apply. A possible starting point:

In the function signature, arguments which are optional have, in their type signatures, default value assignments, which are used if the placeholder is specified. Here, the equals sign, followed by the value to use as a default, is used for this purpose, somewhat like an assignment statement.

fn increment(value: &mut i64, by: i64 = 1) {
    // If a _ is put in for by, then 1 is used for by
    *value += by
}

Then, in the caller, instead of being omitted entirely, a distinctive placeholder is used for arguments for which the default value should be used. Here, the placeholder used is _, in similar fashion to dummy values in patterns.

let mut target = 0;
increment(&mut target, _); 
// _ is a placeholder, which is filled in with 1 
// in accordance with increment's argument defaults
assert_eq!(target, 1);

If a different value is desired for an optional argument, it can be used instead of the placeholder at no additional syntactic cost.

increment(&mut target, 2); 
assert_eq!(target, 3);

It makes sense to have optional arguments after mandatory arguments here, unlike in some other ideas for optional arguments, since the placeholders make explicit which specific arguments are omitted and are intended to be filled in with defaults.

fn f(a: i64, b: i64 = 4, c: i64) { /* irrelevant */ }
// In user code...
f(3, 4); // Wrong number of arguments
f(3, 4, _); // Optional placeholder in mandatory argument position
f(3, _, 4); // Okay, equivalent to f(3, 4, 4)

Changing a mandatory argument to be optional later is also backwards compatible (but changing the default value or changing the argument back to a mandatory one is still breaking).

// Much later...
fn f(a: i64, bee: i64 = 4, c: i64 = 6) { /* irrelevant */ }
// After updates... 
f(3, 4, _); // Now this is okay, and is equivalent to f(3, 4, 6)
f(3, _, 4); // Still okay, and still equivalent to f(3, 4, 4)
2 Likes
19

I'm pretty confident it should be opt-in, not opt-out. There's a semantic difference between an optional parameter (i.e. one that doesn't have to be supplied) and a value of type Option<T> which is a mandatory parameter. Conflating this two would not only cause confusion, but would also reduce the usefulness of default parameters because it's possible that sometimes a default of None doesn't make sense.

Incidentally, I don't think defaulting to Default::default() (wow, lots of defaults in this sentence) is a good decision either. Most languages with default parameter values allow the author of a function to specify the default value explicitly only. And in Rust's case, it could be similar:

fn transmogrify(gork: Foo = Default::default()) {
    …
}

It could be specified that the default value is the instance returned by the Default impl, but it wouldn't be restricted to that. This has the added benefit that it syntactically disambiguates optional/default parameters (regardless of the concrete syntax used, the compiler could distinguish between the absence and presence of a default value specifier). So it probably eliminates or at least greatly reduces the needs for additional annotation noise.

Coming from Swift, a language with pervasive use of named parameters, I'm quite sure it would be a breaking change, and even if it weren't, it would be a huge pain point. Swift does almost exactly the same: parameter names are part of the function name, and it causes all sorts of not-so-edge cases that are painful to learn and get over when it doesn't compile. (At least I haven't seen it lead to bugs.)

A side question: would this overloading work based on the function name and parameter names only?

1 Like
20

Yes, I was kind of spit-balling here on the "possibilities", but, as you and others have pointed out, this would just result in far too much magic happening for little benefit. Opt-In with Explicit Defaults is definitely the preferred solution.

I'm not sure that it would have to be a breaking change. As I mentioned, if you allow that the first definition of a function of a particular name within a scope does not include the parameter names as part of the mangled function name, then, definitions of the same function name after that in lexical order could have mangling with the names of the parameters, this would allow adding new overloads to an existing API with different named parameter sets in a 100% backwards compatible fashion (as far as I can tell).

Yes, what I am proposing is overloading based on parameter names rather than types. So, you could have 2 overloads with the same types but different parameter names, but, not 2 overloads with the same set of parameter names. The parameter names become part of the mangled function/method name, so, in reality, there is no real overloading, all function/method names that the compiler ultimately deals with are unique.

1 Like