Extending implicit coercion of function items to function pointers during trait resolution

Implicit coercion already happens in some cases, but trait resolution doesn't fall back to consider coercions.

This issue shows up in various forms on the issue tracker (some examples #58078 #62385 #62621 #86654 #95863), but I couldn't find a more structured discussion around adding support for this.

The main problem is visible in this example:

trait A {}

impl A for fn() {}

fn a() {}

fn test<T: A>(_: T) {}

fn main() {
    test(a); // Fails to compile
    test(a as fn()); // Compiles
}

As you can see, there are some workarounds for it, including the explicit cast to as fn().

For my use-case, I rely a lot on function pointers, casting them to WebAssembly table indexes and sharing them between the host and guest environments. This is only possible with function pointers, so I can't depend on some workarounds with closures (Fn* traits). I need to be able to do a fn as usize and get the pointer.The coercion of non-capturing closure -> fn pointer works already and creates nice ergonomics for me, but because this trait coercion doesn't happen automatically, I can't use traits to create some higher level abstractions. At the moment my code is also full of * as fn(_,_,_,_) -> _ and * as fn(_,_) -> _ casts making it a bit ugly.

The error messages around this can also be confusing:

   |
10 |     test(a); // Fails to compile
   |     ---- ^ the trait `A` is not implemented for `fn() {a}`
   |     |
   |     required by a bound introduced by this call
   |
   = help: the trait `A` is implemented for `fn()`

First time I came across this, I didn't understand the significance of the {a} part and it looked like the same type. I would also think that fn() {a} is a more specific version of fn() and that this should just work.

I can't think of any drawbacks or ambiguity in adding implicit coercion in this case. It could also be that I'm missing something obvious?

This would significantly improve Rust ergonomics for my use-case and I would love to see it supported.

I suppose one hiccup would be if there are multiple different types that the argument could coerce to that would satisfy the trait bounds, including new coercions that might be added to the language in the future.

1 Like

Without going into detail on this, I can imagine a lot of potential ambiguity problems with something like this proposal.

I feel like you're proposing an overly complicated solution here, the easier approach would be if there was a trait for "type that can be coerced into function pointer", and then you could just make your consuming function more generic.

Admitted, even that approach would only work for your use case if we could rule out types that can coerce into into different function pointer types of different arity. (I assume your use case is that you support functions of different arity, or perhaps I misunderstood... why is there a trait involved in the first place?)

For the time being, if you see yourself commonly using as fn(_, _) -> _, then perhaps redesign your API to be less generic and have multiple methods each of which expects a concrete function pointer type?

I suppose one hiccup would be if there are multiple different types that the argument could coerce to that would satisfy the trait bounds, including new coercions that might be added to the language in the future.

From my understanding, a function item can only coerce to a function pointer with a matching signature. I can't think of an example where a function would be coerced to something else in the future, but if this ever happens it could require an explicit cast to resolve the ambiguity.

Admitted, even that approach would only work for your use case if we could rule out types that can coerce into into different function pointer types of different arity. (I assume your use case is that you support functions of different arity, or perhaps I misunderstood... why is there a trait involved in the first place?)

Just to be clear, this wasn't supposed to be a discussion about my specific use-case. I feel like this is a common issue that people run into. Recently I saw it pop up in Jon's video. I think his reaction to it was similar to what many people do when they run into it. It's a bit confusing, you try to match the lifetimes first, then when nothing works you give up and resort to a macro in the end. At no point it's obvious that you would need to add as fn for it to work.

Adding that trait would solve the issue for me, but it wouldn't be of any help for most people that run the first time into it. However, coercion would make it just work for everyone. It might be a bit more complicated, but it also eliminates many more issues.

Improving the diagnostics is certainly good. But I don't think we should automatically coerce here. Rather, I want to be able to implement a trait for any function (be generic over a generic Fn(T)).

1 Like

Haven’t watched through the relevant section fully yet, but if avoiding confusion is the main goal, a diagnostic that suggests adding an as fn(…) -> … cast might be enough. Furthermore, such a diagnostic could be a good first step anyway in evaluating how reliably the compiler can detect cases like this where coercing to a function pointer type makes a trait bound work, and how reliably the right function pointer type can be inferred… etc.


One additional point of confusion is how function types are written. for<'r> fn(&'r [Guess]) -> String {guess} does hardly indicate clearly enough that “this type is not the same as for<'r> fn(&'r [Guess]) -> String”, even though probably half the time you’ll run into this in error messages, it’ll be absolutely crucial for you to understand that you don’t have a function pointer type.

Maybe it should look more like closure types, something like

[function item `guess`: for<'r> fn(&'r [Guess]) -> String]

while we’re at it, closure types could probably benefit from having a visible signature, too, so something like

[closure@src/lib.rs(112:40 - 112:69): for<'r> fn(&'r [Guess]) -> String]

for a non-capturing closure, and something like

[closure@src/lib.rs(112:40 - 112:69): for<'r> Fn(&'r [Guess]) -> String]

or

[closure@src/lib.rs(112:40 - 112:69): for<'r> FnMut(&'r [Guess]) -> String]

or

[closure@src/lib.rs(112:40 - 112:69): for<'r> FnOnce(&'r [Guess]) -> String]

for capturing closures, depending on what the most general function trait they implement is.

6 Likes

Without the trait indirection, the error messages are a lot clearer about the difference:

error[E0308]: mismatched types
  = note: expected fn pointer `fn(())`
                found fn item `fn(Argument) -> Return {function}`
error[E0308]: mismatched types
  = note: expected fn pointer `fn(())`
                found closure `[closure@src/main.rs:9:21: 9:56]`

And it definitely degrades for traits:

error[E0277]: the trait bound `fn(Argument) -> Return {function}: FnPtr` is not satisfied
  = note: the trait `FnPtr` is not implemented for `fn(Argument) -> Return {function}`
  = help: the trait `FnPtr` is implemented for `fn(Argument) -> Return`
error[E0277]: the trait bound `[closure@src/main.rs:14:11: 14:34]: FnPtr` is not satisfied
  = note: the trait `FnPtr` is not implemented for `[closure@src/main.rs:14:11: 14:34]`
  = help: the trait `FnPtr` is implemented for `fn(Argument) -> Return`

For this specific use case, we already want to introduce a FnPtr trait in std, and making it include fn items and closures (which are ptr coercible) seems reasonable.

For trait implementations that aren't able to be blanket provided for "fn pointer like" fn items and closures, though, this absolutely can and should be improved to specify the type "kind" like with E0308.

To be honest, I even find that supposedly better error message too subtle. Many people won't be able to understand how different “function item” and “function pointer” are, or perhaps not even notice this difference in wording. Unlike the rust-source-code-highlighted example you posted above, the words “pointer” and “item” are not highlighted in bright red color in the actual error message. IMO, the best thing would be if the function item types themself look radically different from function pointer types. Admitted, their current rendering is somewhat fancy and neat-looking, but that's not too valuable because the most important thing when you read such a type is often that you should very clearly notice that it's not a function pointer.

A fundamental problem here is that the function pointer type doesn't really say “pointer” anywhere and looks a lot like a function item; but that ship has sailed since it has a stable syntax. But the way that function item types are displayed is completely arbitrary and can be changed.

3 Likes

(yeah, given a blank check I'd prefer &'static fn(), but actually unloading code is also just nonfunctional anyway. also that's perhaps overly von-neuman biased to treat it as a normal reference-to-unsized where on a harvard architecture like wasm they're a different kind of thing.)

I agree, it's still too subtle. But it's better, and a simple change for to help offer the opportunity to notice what the issue is.

So, for comparison's sake, here's how the types get handled in the -C symbol-mangling-version=v0 scheme:

  • fn ptr
    • demangled: example::f::<fn(example::Argument) -> example::Return>
    • mangled: _RINvCsgMsCBXjeJub_7example1fFNtB2_8ArgumentENtB2_6ReturnEB2_
    • unwrapped: FNtB2_8ArgumentENtB2_6ReturnE
    • F                              # function
       Nt                            # - type
         B2_                         #     example (back-reference)
            8Argument                #     Argument
                     E               #   end
                      Nt             # - type
                        B2_          #     example (back-reference)
                           6Return   #     Return
                                  E  #   end
      
  • fn item
    • demangled: example::f::<example::mono::fn_item>
    • mangled: _RINvCsgMsCBXjeJub_7example1fNvNvB2_4mono7fn_itemEB2_
    • unwrapped: NvNvB2_4mono7fn_itemE
    • Nv                     # value
        Nv                   # - value
          B2_                #     example (back-reference)
             4mono           #     mono
                  7fn_item   #     fn_item
                          E  #   end
      
  • closure
    • demangled: example::f::<example::mono::{closure#0}>
    • mangled: _RINvCsgMsCBXjeJub_7example1fNCNvB2_4mono0EB2_
    • unwrapped: NCNvB2_4mono0E
    • NC              # closure
        Nv            # - value
          B2_         #     example (back-reference)
             4mono    #     mono
                  0   #     0 (unnamed disambiguator)
                   E  #   end
      

So as far as name mangling goes, fn pointers are identified by their arguments/returns, fn items are identified by path, and closures are identified by scope path + disambiguator id.

Also just to make it available for comparison, rustc currently shows

  • fn item fn(Argument) -> Return {f}
  • fn pointer fn(Argument) -> Return
  • closure [closure@src/lib.rs:11:19: 11:56]

Unstructured thoughts:

  • fn pointers' types are in source, so can't (well, shouldn't) be changed.
  • fn items definitely would get confused with pointers less if they put the name first (e.g. f {fn(Argument) -> Return}) instead of after.
  • closures could benefit from including the signature, though that's mitigated by the rest of the common errors[1].
  • using a similar syntax for both could be beneficial
  • sticking in Fn/FnMut/FnOnce into there to indicate how it can be used is good as well
  • but fn vs Fn with no other indication is also way too subtle
  • my proposed bikeshed color:
    [fn function@crate/src/lib.rs(L:C..L:C): fn(A) -> R]
    [fn {closure}@crate/src/lib.rs(L:C..L:C): fn(A) -> R]
    
  • but this may conflate fn item with closures too much?
  • which maybe is okay; it's certainly more correct than conflating items and pointers
  • because closures are function items, just unnamed ones
  • the source location is necessary for disambiguating closures but redundant with the item name path for fn items, so useful but perhaps unneeded

  1. error[E0308]: mismatched types
      --> src/lib.rs:15:20
       |
    11 |     let closure = |_: Argument| -> Return { Return(0) };
       |                   ------------------------------------- the found closure
    ...
    15 |     let _: fn(_) = closure;
       |            -----   ^^^^^^^ expected `()`, found struct `Return`
       |            |
       |            expected due to this
       |
       = note: expected fn pointer `fn(_)`
                     found closure `[closure@src/lib.rs:11:19: 11:56]`
    
    ↩︎
3 Likes

Perhaps I have misjudged complexities here. But I thought changing fn(Foo) -> Bar {name} into something else, say, [function item `name`: fn(Foo) -> Bar], would be an almost trivial change, simpler than almost anything else, presumably just a change in a single formatting string[1] or something like that (though I have admittedly not searched for the actual source code that produces this output yet).

Or course more complex changes, like adding new information (source location) or changing closures, too, (to include a signature) could/would be significantly less trivial; but that's a good subsequent improvement.


  1. plus probably a gazillion test cases changing their output that need to be blessed ↩︎

Maybe it would be but I wouldn't know about that one. I have been in the diagnostics code enough now to know that that one would be a simple change.

What if the compiler included a note, in such type mismatches, that “function items are not function pointers: see <link to error index> for more information”? It would be useful for anyone running in to this to have the opportunity to read about what a function item is, and how it can and can't be used. Even if they don't follow the link, they've at least been informed that a “function item” is something specifically important and not just some quirk of the compiler's error description generation not calling the latter a pointer too.

4 Likes

That's certainly a useful addition in any case.

@CAD97

For this specific use case, we already want to introduce a FnPtr trait in std, and making it include fn items and closures (which are ptr coercible) seems reasonable.

This would be the best solution for my specific use case. I actually looked a long time for something like this, but couldn't find any discussions specifically related to it or the intention to add it. The closest I came to it was this workaround mentoined by @steffahn in a post from last year.

EDIT: I found this recent PR: builtin trait to abstract over function pointers by lcnr · Pull Request #99531 · rust-lang/rust · GitHub

@kpreid

What if the compiler included a note, in such type mismatches, that “function items are not function pointers: see for more information”? It would be useful for anyone running in to this to have the opportunity to read about what a function item is, and how it can and can't be used

I think this would be a huge help. Only after I understood what a "function item" was, it was possible for me to dig deeper into this and really understand the issue. And it gives you a better keyword to google for.

I also like the suggested formatting changes to make it more obvious that something is a function item.

The main problem with that is that it's never going to be foolproof to figure out when to print that message, unless you always print it. As soon as you have a trait sightly harder to fulfill than the examples with one implementing type, or perhaps even involve blanket implementations, the compiler provided list of potential types becomes less useful, and people'll ignore the type list and just look at the improper type then try to figure out why it's not implementing the trait.

(I know because I've done pretty much exactly that.)

In order to robustly provide this hint, the compiler'd need to try resolving with the fn pointer type, and if that resolves properly, provide an as fn(...) -> _ hint and help explaining the difference. I've now filed a docs issue for this as well.

I've started trying this out and working on a PR. It's slightly more than changing a single "format string", because apparently there's separate special-case logic for printing two types and highlighting the differences between them. Still straightforward though. But... damn, there's a lot of test cases that need updating :sweat_smile:

Edit: Now the PR is publicly visible. I’m excited about CI results. More distinctive pretty-printing of function item types by steffahn · Pull Request #99927 · rust-lang/rust · GitHub

3 Likes

I found a discussion from 2020 on Zulip around performing this coercion.

I came across it reading this issue and would also like to highlight this comment from it:

I know this issue is about improving the diagnostic, but it seemed as good a place as any to note that fixing this is desirable, but apparently there are a number of implementation challenges, according to @nikomatsakis (Zulip source).

Reading the comments in this thread, I think the consensus shifted a bit today and it is not desirable anymore to have this fixed. Unrelated to the implementation complexity.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.