Extending implicit coercion of function items to function pointers during trait resolution

bkolobara · July 28, 2022, 2:02pm

Implicit coercion already happens in some cases, but trait resolution doesn't fall back to consider coercions.

This issue shows up in various forms on the issue tracker (some examples #58078 #62385 #62621 #86654 #95863), but I couldn't find a more structured discussion around adding support for this.

The main problem is visible in this example:

trait A {}

impl A for fn() {}

fn a() {}

fn test<T: A>(_: T) {}

fn main() {
    test(a); // Fails to compile
    test(a as fn()); // Compiles
}

As you can see, there are some workarounds for it, including the explicit cast to as fn().

For my use-case, I rely a lot on function pointers, casting them to WebAssembly table indexes and sharing them between the host and guest environments. This is only possible with function pointers, so I can't depend on some workarounds with closures (Fn* traits). I need to be able to do a fn as usize and get the pointer.The coercion of non-capturing closure -> fn pointer works already and creates nice ergonomics for me, but because this trait coercion doesn't happen automatically, I can't use traits to create some higher level abstractions. At the moment my code is also full of * as fn(_,_,_,_) -> _ and * as fn(_,_) -> _ casts making it a bit ugly.

The error messages around this can also be confusing:

   |
10 |     test(a); // Fails to compile
   |     ---- ^ the trait `A` is not implemented for `fn() {a}`
   |     |
   |     required by a bound introduced by this call
   |
   = help: the trait `A` is implemented for `fn()`

First time I came across this, I didn't understand the significance of the {a} part and it looked like the same type. I would also think that fn() {a} is a more specific version of fn() and that this should just work.

I can't think of any drawbacks or ambiguity in adding implicit coercion in this case. It could also be that I'm missing something obvious?

This would significantly improve Rust ergonomics for my use-case and I would love to see it supported.

Jules-Bertholet · July 28, 2022, 2:28pm

I suppose one hiccup would be if there are multiple different types that the argument could coerce to that would satisfy the trait bounds, including new coercions that might be added to the language in the future.

steffahn · July 28, 2022, 2:30pm

Without going into detail on this, I can imagine a lot of potential ambiguity problems with something like this proposal.

I feel like you're proposing an overly complicated solution here, the easier approach would be if there was a trait for "type that can be coerced into function pointer", and then you could just make your consuming function more generic.

Admitted, even that approach would only work for your use case if we could rule out types that can coerce into into different function pointer types of different arity. (I assume your use case is that you support functions of different arity, or perhaps I misunderstood... why is there a trait involved in the first place?)

For the time being, if you see yourself commonly using as fn(_, _) -> _, then perhaps redesign your API to be less generic and have multiple methods each of which expects a concrete function pointer type?

bkolobara · July 28, 2022, 3:37pm

I suppose one hiccup would be if there are multiple different types that the argument could coerce to that would satisfy the trait bounds, including new coercions that might be added to the language in the future.

From my understanding, a function item can only coerce to a function pointer with a matching signature. I can't think of an example where a function would be coerced to something else in the future, but if this ever happens it could require an explicit cast to resolve the ambiguity.

Admitted, even that approach would only work for your use case if we could rule out types that can coerce into into different function pointer types of different arity. (I assume your use case is that you support functions of different arity, or perhaps I misunderstood... why is there a trait involved in the first place?)

Just to be clear, this wasn't supposed to be a discussion about my specific use-case. I feel like this is a common issue that people run into. Recently I saw it pop up in Jon's video. I think his reaction to it was similar to what many people do when they run into it. It's a bit confusing, you try to match the lifetimes first, then when nothing works you give up and resort to a macro in the end. At no point it's obvious that you would need to add as fn for it to work.

Adding that trait would solve the issue for me, but it wouldn't be of any help for most people that run the first time into it. However, coercion would make it just work for everyone. It might be a bit more complicated, but it also eliminates many more issues.

chrefr · July 28, 2022, 3:46pm

Improving the diagnostics is certainly good. But I don't think we should automatically coerce here. Rather, I want to be able to implement a trait for any function (be generic over a generic Fn(T)).

steffahn · July 28, 2022, 3:49pm

Haven’t watched through the relevant section fully yet, but if avoiding confusion is the main goal, a diagnostic that suggests adding an as fn(…) -> … cast might be enough. Furthermore, such a diagnostic could be a good first step anyway in evaluating how reliably the compiler can detect cases like this where coercing to a function pointer type makes a trait bound work, and how reliably the right function pointer type can be inferred… etc.

One additional point of confusion is how function types are written. for<'r> fn(&'r [Guess]) -> String {guess} does hardly indicate clearly enough that “this type is not the same as for<'r> fn(&'r [Guess]) -> String”, even though probably half the time you’ll run into this in error messages, it’ll be absolutely crucial for you to understand that you don’t have a function pointer type.

Maybe it should look more like closure types, something like

[function item `guess`: for<'r> fn(&'r [Guess]) -> String]

while we’re at it, closure types could probably benefit from having a visible signature, too, so something like

[closure@src/lib.rs(112:40 - 112:69): for<'r> fn(&'r [Guess]) -> String]

for a non-capturing closure, and something like

[closure@src/lib.rs(112:40 - 112:69): for<'r> Fn(&'r [Guess]) -> String]

or

[closure@src/lib.rs(112:40 - 112:69): for<'r> FnMut(&'r [Guess]) -> String]

or

[closure@src/lib.rs(112:40 - 112:69): for<'r> FnOnce(&'r [Guess]) -> String]

for capturing closures, depending on what the most general function trait they implement is.

CAD97 · July 28, 2022, 9:40pm

Without the trait indirection, the error messages are a lot clearer about the difference:

error[E0308]: mismatched types
  = note: expected fn pointer `fn(())`
                found fn item `fn(Argument) -> Return {function}`
error[E0308]: mismatched types
  = note: expected fn pointer `fn(())`
                found closure `[closure@src/main.rs:9:21: 9:56]`

And it definitely degrades for traits:

error[E0277]: the trait bound `fn(Argument) -> Return {function}: FnPtr` is not satisfied
  = note: the trait `FnPtr` is not implemented for `fn(Argument) -> Return {function}`
  = help: the trait `FnPtr` is implemented for `fn(Argument) -> Return`
error[E0277]: the trait bound `[closure@src/main.rs:14:11: 14:34]: FnPtr` is not satisfied
  = note: the trait `FnPtr` is not implemented for `[closure@src/main.rs:14:11: 14:34]`
  = help: the trait `FnPtr` is implemented for `fn(Argument) -> Return`

For this specific use case, we already want to introduce a FnPtr trait in std, and making it include fn items and closures (which are ptr coercible) seems reasonable.

For trait implementations that aren't able to be blanket provided for "fn pointer like" fn items and closures, though, this absolutely can and should be improved to specify the type "kind" like with E0308.

github.com/rust-lang/rust

Trait bound not satisfied should specify "fn item" vs "fn ptr" kind

opened 09:38PM - 28 Jul 22 UTC

CAD97

A-diagnostics T-compiler

Given the following code: [[playground]](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=b14b884b40223c3864d9c4ebabce0f4d) ```rust struct Argument; struct Return; fn function(_: Argument) -> Return { todo!() } trait Trait {} impl Trait for fn(Argument) -> Return {} fn takes(_: impl Trait) {} fn main() { takes(function); takes(|_: Argument| -> Return { todo!() }); } ``` The current output is: ``` error[E0277]: the trait bound `fn(Argument) -> Return {function}: Trait` is not satisfied --> src/main.rs:12:11 | 12 | takes(function); | ----- ^^^^^^^^ the trait `Trait` is not implemented for `fn(Argument) -> Return {function}` | | | required by a bound introduced by this call | = help: the trait `Trait` is implemented for `fn(Argument) -> Return` note: required by a bound in `takes` --> src/main.rs:9:18 | 9 | fn takes(_: impl Trait) {} | ^^^^^ required by this bound in `takes` error[E0277]: the trait bound `[closure@src/main.rs:13:11: 13:34]: Trait` is not satisfied --> src/main.rs:13:5 | 13 | takes(|_: Argument| -> Return { todo!() }); | ^^^^^ the trait `Trait` is not implemented for `[closure@src/main.rs:13:11: 13:34]` | = help: the trait `Trait` is implemented for `fn(Argument) -> Return` note: required by a bound in `takes` --> src/main.rs:9:18 | 9 | fn takes(_: impl Trait) {} | ^^^^^ required by this bound in `takes` ``` Compare E0308 mismatched types: [[playground]](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=2db6dd0f0533bd0c07d70573c507f57e) ``` error[E0308]: mismatched types --> src/main.rs:12:21 | 12 | let _: fn(()) = function; | ------ ^^^^^^^^ expected `()`, found struct `Argument` | | | expected due to this | = note: expected fn pointer `fn(())` found fn item `fn(Argument) -> Return {function}` error[E0308]: mismatched types --> src/main.rs:13:21 | 13 | let _: fn(()) = |_: Argument| -> Return { todo!() }; | ------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `()`, found struct `Argument` | | | expected due to this | = note: expected fn pointer `fn(())` found closure `[closure@src/main.rs:13:21: 13:44]` ``` Ideally the output (of the reported example) should look like: ``` error[[E0277]](https://doc.rust-lang.org/nightly/error-index.html#E0277): the trait bound `fn(Argument) -> Return {function}: Trait` is not satisfied --> src/main.rs:12:11 | 12 | takes(function); | ----- ^^^^^^^^ the trait `Trait` is not implemented for fn item `fn(Argument) -> Return {function}` | | | required by a bound introduced by this call | = help: the trait `Trait` is implemented for fn pointer `fn(Argument) -> Return` note: required by a bound in `takes` --> src/main.rs:9:18 | 9 | fn takes(_: impl Trait) {} | ^^^^^ required by this bound in `takes` error[[E0277]](https://doc.rust-lang.org/nightly/error-index.html#E0277): the trait bound `[closure@src/main.rs:13:11: 13:34]: Trait` is not satisfied --> src/main.rs:13:5 | 13 | takes(|_: Argument| -> Return { todo!() }); | ^^^^^ the trait `Trait` is not implemented for closure `[closure@src/main.rs:13:11: 13:34]` | = help: the trait `Trait` is implemented for fn pointer `fn(Argument) -> Return` note: required by a bound in `takes` --> src/main.rs:9:18 | 9 | fn takes(_: impl Trait) {} | ^^^^^ required by this bound in `takes` ```

steffahn · July 28, 2022, 10:22pm

To be honest, I even find that supposedly better error message too subtle. Many people won't be able to understand how different “function item” and “function pointer” are, or perhaps not even notice this difference in wording. Unlike the rust-source-code-highlighted example you posted above, the words “pointer” and “item” are not highlighted in bright red color in the actual error message. IMO, the best thing would be if the function item types themself look radically different from function pointer types. Admitted, their current rendering is somewhat fancy and neat-looking, but that's not too valuable because the most important thing when you read such a type is often that you should very clearly notice that it's not a function pointer.

A fundamental problem here is that the function pointer type doesn't really say “pointer” anywhere and looks a lot like a function item; but that ship has sailed since it has a stable syntax. But the way that function item types are displayed is completely arbitrary and can be changed.

CAD97 · July 29, 2022, 12:12am

(yeah, given a blank check I'd prefer &'static fn(), but actually unloading code is also just nonfunctional anyway. also that's perhaps overly von-neuman biased to treat it as a normal reference-to-unsized where on a harvard architecture like wasm they're a different kind of thing.)

I agree, it's still too subtle. But it's better, and a simple change for to help offer the opportunity to notice what the issue is.

So, for comparison's sake, here's how the types get handled in the -C symbol-mangling-version=v0 scheme:

fn ptr

demangled: example::f::<fn(example::Argument) -> example::Return>
mangled: _RINvCsgMsCBXjeJub_7example1fFNtB2_8ArgumentENtB2_6ReturnEB2_
unwrapped: FNtB2_8ArgumentENtB2_6ReturnE

F                              # function
 Nt                            # - type
   B2_                         #     example (back-reference)
      8Argument                #     Argument
               E               #   end
                Nt             # - type
                  B2_          #     example (back-reference)
                     6Return   #     Return
                            E  #   end

fn item

demangled: example::f::<example::mono::fn_item>
mangled: _RINvCsgMsCBXjeJub_7example1fNvNvB2_4mono7fn_itemEB2_
unwrapped: NvNvB2_4mono7fn_itemE

Nv                     # value
  Nv                   # - value
    B2_                #     example (back-reference)
       4mono           #     mono
            7fn_item   #     fn_item
                    E  #   end

closure

demangled: example::f::<example::mono::{closure#0}>
mangled: _RINvCsgMsCBXjeJub_7example1fNCNvB2_4mono0EB2_
unwrapped: NCNvB2_4mono0E

NC              # closure
  Nv            # - value
    B2_         #     example (back-reference)
       4mono    #     mono
            0   #     0 (unnamed disambiguator)
             E  #   end

So as far as name mangling goes, fn pointers are identified by their arguments/returns, fn items are identified by path, and closures are identified by scope path + disambiguator id.

Also just to make it available for comparison, rustc currently shows

fn item fn(Argument) -> Return {f}
fn pointer fn(Argument) -> Return
closure [closure@src/lib.rs:11:19: 11:56]

Unstructured thoughts:

fn pointers' types are in source, so can't (well, shouldn't) be changed.
fn items definitely would get confused with pointers less if they put the name first (e.g. f {fn(Argument) -> Return}) instead of after.
closures could benefit from including the signature, though that's mitigated by the rest of the common errors^[1].
using a similar syntax for both could be beneficial
sticking in Fn/FnMut/FnOnce into there to indicate how it can be used is good as well
but fn vs Fn with no other indication is also way too subtle

my proposed bikeshed color:

[fn function@crate/src/lib.rs(L:C..L:C): fn(A) -> R]
[fn {closure}@crate/src/lib.rs(L:C..L:C): fn(A) -> R]

but this may conflate fn item with closures too much?
which maybe is okay; it's certainly more correct than conflating items and pointers
because closures are function items, just unnamed ones
the source location is necessary for disambiguating closures but redundant with the item name path for fn items, so useful but perhaps unneeded

error[E0308]: mismatched types
  --> src/lib.rs:15:20
   |
11 |     let closure = |_: Argument| -> Return { Return(0) };
   |                   ------------------------------------- the found closure
...
15 |     let _: fn(_) = closure;
   |            -----   ^^^^^^^ expected `()`, found struct `Return`
   |            |
   |            expected due to this
   |
   = note: expected fn pointer `fn(_)`
                 found closure `[closure@src/lib.rs:11:19: 11:56]`

↩︎

steffahn · July 29, 2022, 12:32am

Perhaps I have misjudged complexities here. But I thought changing fn(Foo) -> Bar {name} into something else, say, [function item `name`: fn(Foo) -> Bar], would be an almost trivial change, simpler than almost anything else, presumably just a change in a single formatting string^[1] or something like that (though I have admittedly not searched for the actual source code that produces this output yet).

Or course more complex changes, like adding new information (source location) or changing closures, too, (to include a signature) could/would be significantly less trivial; but that's a good subsequent improvement.

plus probably a gazillion test cases changing their output that need to be blessed ↩︎

CAD97 · July 29, 2022, 2:00am

Maybe it would be but I wouldn't know about that one. I have been in the diagnostics code enough now to know that that one would be a simple change.

kpreid · July 29, 2022, 5:35am

What if the compiler included a note, in such type mismatches, that “function items are not function pointers: see <link to error index> for more information”? It would be useful for anyone running in to this to have the opportunity to read about what a function item is, and how it can and can't be used. Even if they don't follow the link, they've at least been informed that a “function item” is something specifically important and not just some quirk of the compiler's error description generation not calling the latter a pointer too.

steffahn · July 29, 2022, 8:48am

That's certainly a useful addition in any case.

bkolobara · July 29, 2022, 10:05am

@CAD97

For this specific use case, we already want to introduce a FnPtr trait in std, and making it include fn items and closures (which are ptr coercible) seems reasonable.

This would be the best solution for my specific use case. I actually looked a long time for something like this, but couldn't find any discussions specifically related to it or the intention to add it. The closest I came to it was this workaround mentoined by @steffahn in a post from last year.

EDIT: I found this recent PR: builtin trait to abstract over function pointers by lcnr · Pull Request #99531 · rust-lang/rust · GitHub

@kpreid

What if the compiler included a note, in such type mismatches, that “function items are not function pointers: see for more information”? It would be useful for anyone running in to this to have the opportunity to read about what a function item is, and how it can and can't be used

I think this would be a huge help. Only after I understood what a "function item" was, it was possible for me to dig deeper into this and really understand the issue. And it gives you a better keyword to google for.

I also like the suggested formatting changes to make it more obvious that something is a function item.

CAD97 · July 29, 2022, 8:49pm

The main problem with that is that it's never going to be foolproof to figure out when to print that message, unless you always print it. As soon as you have a trait sightly harder to fulfill than the examples with one implementing type, or perhaps even involve blanket implementations, the compiler provided list of potential types becomes less useful, and people'll ignore the type list and just look at the improper type then try to figure out why it's not implementing the trait.

(I know because I've done pretty much exactly that.)

In order to robustly provide this hint, the compiler'd need to try resolving with the fn pointer type, and if that resolves properly, provide an as fn(...) -> _ hint and help explaining the difference. I've now filed a docs issue for this as well.

github.com/rust-lang/rust

Unsatisfied trait bound on fn item should suggest `as fn()` if it is satisfied for fn pointer

opened 08:47PM - 29 Jul 22 UTC

CAD97

A-diagnostics T-compiler

Given the following code: [[playground]](https://play.rust-lang.org/?version=sta…ble&mode=debug&edition=2021&gist=47c96512e5b66aac5fab70201ece0d87) ```rust trait Trait {} impl Trait for fn() {} fn take(_: impl Trait) {} fn f() {} fn main() { take(f); } ``` The current output is: ``` error[[E0277]](https://doc.rust-lang.org/stable/error-index.html#E0277): the trait bound `fn() {f}: Trait` is not satisfied --> src/main.rs:7:10 | 7 | take(f); | ---- ^ the trait `Trait` is not implemented for `fn() {f}` | | | required by a bound introduced by this call | = help: the trait `Trait` is implemented for `fn()` note: required by a bound in `take` --> src/main.rs:3:17 | 3 | fn take(_: impl Trait) {} | ^^^^^ required by this bound in `take` ``` Ideally the output should look like: ``` error[[E0277]](https://doc.rust-lang.org/stable/error-index.html#E0277): the trait bound `fn() {f}: Trait` is not satisfied --> src/main.rs:7:10 | 7 | take(f); | ---- ^ the trait `Trait` is not implemented for fn item `fn() {f}` | | | required by a bound introduced by this call | = help: the trait `Trait` is implemented for fn pointer `fn()` note: required by a bound in `take` --> src/main.rs:3:17 | 3 | fn take(_: impl Trait) {} | ^^^^^ required by this bound in `take` | help: Convert the fn item to a fn pointer with `as` | 7 | take(f as fn()); | ++++++++ ``` <details><summary>A more involved diagnostic example:</summary> [[playground]](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=56c1021f84a0619534292ef1f3c19b65) ```rust trait Trait {} impl<A: PartialEq> Trait for A {} fn take(_: impl Trait) {} fn f() {} fn main() { take(f); } ``` The current output is: ``` error[[E0277]](https://doc.rust-lang.org/nightly/error-index.html#E0277): can't compare `fn() {f}` with `fn() {f}` --> src/main.rs:7:10 | 7 | take(f); | ---- ^ no implementation for `fn() {f} == fn() {f}` | | | required by a bound introduced by this call | = help: the trait `PartialEq` is not implemented for `fn() {f}` = help: the following other types implement trait `PartialEq<Rhs>`: extern "C" fn() -> Ret extern "C" fn(A, B) -> Ret extern "C" fn(A, B, ...) -> Ret extern "C" fn(A, B, C) -> Ret extern "C" fn(A, B, C, ...) -> Ret extern "C" fn(A, B, C, D) -> Ret extern "C" fn(A, B, C, D, ...) -> Ret extern "C" fn(A, B, C, D, E) -> Ret and 68 others note: required because of the requirements on the impl of `Trait` for `fn() {f}` --> src/main.rs:2:20 | 2 | impl<A: PartialEq> Trait for A {} | ^^^^^ ^ note: required by a bound in `take` --> src/main.rs:3:17 | 3 | fn take(_: impl Trait) {} | ^^^^^ required by this bound in `take` ``` Ideally the output should look like: ``` error[[E0277]](https://doc.rust-lang.org/nightly/error-index.html#E0277): can't compare `fn() {f}` with `fn() {f}` --> src/main.rs:7:10 | 7 | take(f); | ---- ^ no implementation for `fn() {f} == fn() {f}` | | | required by a bound introduced by this call | = help: the trait `PartialEq` is not implemented for fn item `fn() {f}` = help: the trait `PartialEq` is implemented for fn pointer `fn()` note: required because of the requirements on the impl of `Trait` for `fn() {f}` --> src/main.rs:2:20 | 2 | impl<A: PartialEq> Trait for A {} | ^^^^^ ^ note: required by a bound in `take` --> src/main.rs:3:17 | 3 | fn take(_: impl Trait) {} | ^^^^^ required by this bound in `take` | help: Convert the fn item to a fn pointer with `as` | 7 | take(f as fn()); | ++++++++ ``` (bonus side effect: hiding the giant list of explicit variadic fn pointer impls) </details>

steffahn · July 29, 2022, 9:59pm

I've started trying this out and working on a PR. It's slightly more than changing a single "format string", because apparently there's separate special-case logic for printing two types and highlighting the differences between them. Still straightforward though. But... damn, there's a lot of test cases that need updating

Edit: Now the PR is publicly visible. I’m excited about CI results. More distinctive pretty-printing of function item types by steffahn · Pull Request #99927 · rust-lang/rust · GitHub

bkolobara · August 2, 2022, 11:09am

I found a discussion from 2020 on Zulip around performing this coercion.

I came across it reading this issue and would also like to highlight this comment from it:

I know this issue is about improving the diagnostic, but it seemed as good a place as any to note that fixing this is desirable, but apparently there are a number of implementation challenges, according to @nikomatsakis (Zulip source).

Reading the comments in this thread, I think the consensus shifted a bit today and it is not desirable anymore to have this fixed. Unrelated to the implementation complexity.

system · October 31, 2022, 11:09am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
[Feature request] Function items should coerce to function pointers when used as a receiver language design	5	638	February 3, 2022
Implementing traits for function types language design	6	3212	March 25, 2019
[Pre-RFC] Fn item deref language design	9	923	September 19, 2019
Pre-RFC: enable deref coercion on trait-bound parameters language design	2	773	May 25, 2023
Inconsistency: deref coercion doesn't work language design	14	1183	January 20, 2020

Extending implicit coercion of function items to function pointers during trait resolution

Related Topics