Idea: Borrowck Transparent Function Calls

language design
1

This is mostly a vibe-check on the idea.

Motivation

Basically, this idea is meant to allow following usage

impl Foo {
    pub fn wham(&mut self) {
        for baz in &mut self.baz {
            if self.has_bar() {
//             ^^^^ error with current rust
                *baz += "1";
            }

            if let Some(bar) = self.first_bar_mut() {
//                             ^^^^ error with current rust 
                *bar += baz;
            }
        }
    }

    fn has_bar(&self) -> bool {
        !self.bar.is_empty()
    }

    fn first_bar_mut(&mut self) -> Option<&mut String> {
        self.bar.first_mut()
    }
}

This comes up a lot when writing helper functions. And currently you need to either:

  • Somehow split you struct into smaller structs. But
    1. This is tedious since it will impact basically all your code.
    2. It's not always possible to do it (reasonably).
  • Just inline the function manually.
    1. This can get quite verbose and unmaintable.
    2. As soon as helper function body is longer than 5 tokens, the readablity suffers.

Idea

Transparent function calls (TFC) let borrow-checker see through private function calls, using it's body to borrow check instead of it's signature.

..and private function calls could means one of below:

  • All calls to all private functions.
  • All calls to private functions with a certain attribute (#[bikesheld_transparent]) annotated. (This is my perference.)
  • All calls to any self crate defined functions with a certain attribute annotated.
  • Add a new function calls syntax to private functions.

Maybe there's some more variants, but the core idea is downstream crate can't see through, so there's no accidental public api breakage.

The called function itself is still a regular function.

Difference to Previous Similiar Ideas

Following ideas also solves the problem showed in the motivation part. The main difference is that, transparent function calls are much more limited, and introduce minimal new syntax.

View Types

View types is a better solution for public facing apis or traits, but it introduce much more complexity to the type system, so it's inheritly harder to move forward. And we really don't want to annotate view type for helper functions, for obvious reasons (versatility, verbosity, etc.).

Macro Functions

Macro functions's scope is much larger, and is somewhat controversial due to it's template-esque nature. TFCs are not intended to support any duck typing use cases, or any type inference.

Lifetime inference for non-public items

Actually the motivation of this transparent function calls post. And the most similiar one. Core difference is tfc doesn't try to infer anything, and only affect function items.

Misc details

  • To handle recursive TFC, I imagine some kind of inlining happening at MIR level. Haven't think through all these but I think it's doable.
  • Whatever the restriction is, TFC cannot happen across crate.
  • Whether trait methods of local type could be called transparently is undecided yet. I lean on yes if the implementation is local as well, but it's complicated.

Further Extensions

These extensions are what I think is reasonable to add as well. But the TFC can live without them. So I did include them in the main idea.

#[bikesheld_force_transparent]

You can annotate a private (or non-pub?) function with #[bikesheld_force_transparent]. This makes the function not implementing Fn family traits, and is always called transparently. Example:

#[bikesheld_force_transparent]
fn longest(left: &str, right: &str) -> &str {
    if left.len() > right.len() {
        left
    } else {
        right
    }
}

Note that since it's always called transparently, you could omit lifetime constraints on signature.

Variadic Lifetime Parameter Elision Syntax

Introduce '.. (bikesheld syntax) as a shorthand for any replication of '_ parameter in function signature. Examples:

fn foo(r: Ref<'.., T>) {} // Ref<'a, T>
fn bar(c: Ctx<'..>) {} // Ctx<'env, 'scope>
fn bad1(a: Ctx<'env, '..>) // Error: `'..` cannot be used along side other litime
fn bad2(a: Ctx<'.., 'scope>) // Error: `'..` cannot be used along side other litime
fn degenerate(s: String<'..>, v: Vec<'.., i32>) // Ok. There's zero `'_`.

This is meant to solve "add a reference field in one struct, every function got infested" problem. Maybe it deserve a seperate RFC.

End

This is a rough idea that I've had for quite a bit of time. Is this a total no-go? Or it's a too limited to be useful? Or it should add even more limitations? I'd love to hear some feedback!

7 Likes
2

Interesting.

So, we've been talking for a while about supporting structures where the borrows of different methods don't conflict, for public methods. And for public methods, we need something with a well-defined interface that doesn't depend on the implementation details of the method. For that, we've talked about having a way to declare named non-overlapping subsets of the structure, where a method says explicitly which subsets it borrows. View types are one example of this.

However, you're talking about non-public methods, and a mechanism that doesn't cross crate boundaries. For that, the idea of a more automatic, less explicitly declared mechanism seems potentially reasonable. At the very least, it's worth considering.

2 Likes
3

I disagree: it is a strength of rust that the function signature is the border of reasoning (in both directions). It helps bound the amount of work the compiler has to do, which helps compilation time. It also helps bound the amount of things a human has to keep in their head. Finally it helps with compile errors, keeping the error near the issue (C++ templates is an example of where this doesn't happen and the errors are awful).

That is a strength I would hate to loose. I want to see a solution to this issue, but it need to be an explicit solution, not an implicit one.

4

To be clear, I'm not saying it's the right solution. I'm saying it's worth considering.

My first instinct is that we should use the same named-subsets mechanism for both private and public functions. But for private helper functions, automatically detecting which subsets they use would not necessarily be a maintenance hazard. Hence, worth considering, and evaluating the trade-offs.

5

I strongly believe that the ability to reason just from the function signatures and the better error messages is desirable even for private functions. Which means that this proposal is a maintenance hazard (indirectly), since it makes the code base harder to reason about.

The possible impact on compile times doesn't care about private vs public.

5 Likes
6

I think the same thing about this as when essentially the same thing came up 2 days ago

2 Likes
7

Makes sense; then, we could support writing something like &mut{_} self to get a compiler error with a suggestion that fills in the subsets in the {}.

8

I can understand the "ability to reason" argument, and I agree it's a strong point of Rust. But as I stated in the motivation part, current function signature for helpers is often too restricting. It's a common encounter that trying out a small change needs large refactor, which often leads to refactoring again.

TBF most use cases I have in minds just access 1 or 2 fields of self, and doesn't call other functions transparently, or at most 1 call deep. The reasoning would be nearly trivial.

What's your stance if functions that can be called transparently must be annotated with #[helper]? It's not supposed to mark every private function with #[helper]. You could think this as softer and checked escape hatch than unsafe blocks, or a much more ide-friendly version of macro.

On compile times: I believe borrowck basically takes no time (<1%) compared to other part of the compilation process. So I won't worry too much about it.

1 Like
9

That would certainly help for human reasoning, by raising a flag that something funny is going on.

I have found Rust one of the easiest languages to refactor in, as the compiler will tell me what is left to do, rather than me having to hope I got everything (Python) or decode inscrutable error messages (C++ with templates).

Nor I have found it hard to do small experiments, rather when that happens it seems to indicate that my code isn't modular enough or written too much in an OOP style, and thus needs a refactor anyway to a more modular and functional style. Thus I'm not at all convinced this is a real problem.

What I do agree would be useful is partial borrowing. But I suspect you would find an explicit version of it works just fine, especially if it is just depth 0 or 1 as you describe. Then you basically only need to update in one or two places for your experiments. I think we should aim to get explicit partial borrowing first and see how that works out.

Would it stay that way?

10

Bike shedding

What about making it fn foo(*self)? To basically say that the function doesn't borrow self at all. Because if we have &self, there is a lifetime, butby proposal it should be totally ignored, so it has no sense to have it there.

With this, it will no longer be a "full function", and we can stop caring about "function is the reasoning boundary"? (Because it's not a "full function", but basically an extremely restrained macro, with good lsp support).

11

I kind of expected this response. IMO most time it's self-inflicted wound, you won't need the refactor in begin with if you are not using Rust. This cause serious momentum issue, esp in gamedev. Maybe it's suprising to you, but absolute cleanness and correctness is not always the top priority.

:slightly_smiling_face: Ok, disclaimer first: I'm totally not hating or bashing on t-types or t-lang, they do fantastic works.

But here's the thing, partial borrowing (or we call them view types) absolutely won't come out in 3 years, it has much higher language semantic implications. TFC is a much smaller proposal, I can implement it myself in like 2 months. "Wait for view types" is basically equivalent to "forget about this" to me.

This is one thing I have high confidence with an answer of yes.

1 Like
12

I think this is basically equivalent to requiring a marker annotation (#[helper]). And you might still have lifetime parameter in other arguments or in the return type anyway.

13

I meant a lifetime parameter bound to &self. In case of #[helper] is is still technically there, and it would be really awkward to explain. But it's just bike shedding.

14

I think we all agree that view types are the best long-term solution. We can also assume that the explicitness of explicit view type, will in practice not be a real issue because tooling will help to write it.

But in the meantime, could #[annotation] be used, just like try! was a thing in the past? Once view types will be available in stable rust, then cargo fix will be able automatically re-write them and clippy/compiler warning will discourage the use of the deprecated annotation? Basically what I’m proposing is to use an annotation right now for non-public use of partial borrows, while all all code written this way is expected to have to be automatically upgraded to the unified syntax in 2-3 years when partial views are ready and stable.

2 Likes
15

I was kinda hoping we could write nothing, actually, and have it get filled in.

Like today (example inspired from the other thread) if you have https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=b3fc0ce0ebc6c302bd2befd4f35b8522

fn foo(a: &i32, b: &i32, c: &i32) -> &i32 {
    if *c > 0 { a } else { b }
}

Then the error you get is

  = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `a`, `b`, or `c`
help: consider introducing a named lifetime parameter
  |
1 | fn foo<'a>(a: &'a i32, b: &'a i32, c: &'a i32) -> &'a i32 {
  |       ++++     ++          ++          ++          ++

but ideally we could use the MIR borrow-check results to suggest

fn foo<'a>(a: &'a i32, b: &'a i32, c: &i32) -> &'a i32 {

with a note that that's what the body is currently doing.

Though having typed all that I guess that's largely irrelevant to the "view type" part, which yeah, might need a way to say "intentionally wrong; please fix".

That might just be an empty list, though. &mut{} self is probably not actually going to compile, and the error message can tell you which things to put in there that the current implementation needs, with the structured suggestion.

1 Like
16

Agreed that an empty list (or an incorrect list) should be sufficient to get a correction from the compiler.

1 Like
17

I recently thought about syntax like this:

struct Foo {
    foo: usize,
    bar: usize,
}

impl Foo {
    fn foo_mut(Self { foo, .. }: &mut Self) -> &mut usize {
        foo
    }
}

This compiles today. Maybe the borrow checker could infer that only foo is being used here?

6 Likes
18

It would also be interesting to specify mut only for some some subset of fields, and leave others as shared borrows.

1 Like
19

That's true, though that might make borrow checking more complex. Syntactically it'd be easy enough to have (for instance) &{view1, mut view2, mut view3} self. Semantically, we'll have to see if that's worth the added complexity.

20

You could counter that with: if there is a half solution, the need for the full solution gets less and thus it will take even longer since there will be less resources for it. Also, once the full solution is stabilised, what do we do with the half solution? If it was stabilised we can't remove it. Would this idea still be useful to you as a perma-unstable feature that is removed once we get view types?