# Idea: \`with\` block to ensure definite resource release

**URL:** <https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577>\
**Category:** language design\
**Created:** [May 4, 2022, 1:00pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577 "2022-05-04T13:00:15Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 4, 2022, 1:00pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/1 "2022-05-04T13:00:15Z")

</div>

Rust currently use `Drop` to manage resource auto-release. However, it has been proven that this mechanism is unable to protect memory leak or forgetting guards.

Many languages with garbage collections are using `with` blocks to ensure resources will be release after exiting the block.

And it occurs to me that, if we introduce a `with` block to ensure resources or guards can be only allocate in the stack and forbid users to move it, maybe it can ensure that resources or guards will not be forgotten and will be definitely released after exiting the block.

Example: Currently, some apis are already using closures to ensure resources or guards will not be forgotten, such as `std::thread::scope`. It looks like

```rust
thread::scope(|s| {
   // Doing things
});

```

with `with` blocks, it can be written like:

```rust
with let s = "thread::scope_guard" { // scope_guard are a hypothesis name. And the binding syntax is also hypothetical.
  // Doing things
}

// The scope_guard only survives within the block. 
// In the block users can not move it, so it cannot be forgotten.
// After exiting the block, the scope_guard will be released.
// 
// We can also add a new resource management trait/marker, to ensure a guard-like object will only be called with `with` block.

```

With `with` blocks, we can use less closure in these scenario, and maybe provide more information to compiler making it easier to optimize.

I didn't found a similar proposal in internals forum and github issues, sorry if i propose this again. This is a premature idea, however, i think it can be useful.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 4, 2022, 1:51pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/2 "2022-05-04T13:51:42Z")

</div>

What this is doing is basically just stack pinning, a la [`pin_mut!`](https://docs.rs/futures/latest/futures/macro.pin_mut.html).

It's worth noting that if you have

```rust
fn example(f: impl FnOnce()) {
    setup();
    f();
    teardown();
}

example(|| do_stuff());

```

this can be written to the same effect as

```rust
macro example() {
    setup();
    let _defer = OnDrop(|| teardown());
}

example!();
do_stuff();

```

thanks to hygiene.

In general, stack locals _are_ guaranteed to have their destructors run before their memory is reused, so long as they aren't moved from. Introducing a `with` isn't meaningful since `Drop` is already consistently called.

The value of `with` in a GCd language is that with a GC the drop equivalent (usually called collect) isn't called at the end of scope, it's called sometime later when the GC collects the object. `with` effectively introduces an RAII scope to call a cleanup function deterministically at exit.

Plus, `with` would only be useful for soundness if it were _required_ to be used. I don't have a proof, but I suspect that the scoped thread API would be sound if it used `Pin` to guarantee that the scoped thread arena is dropped rather than a closure interface.

---

<div class="post-metadata">

**Author:** ![Lej77](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@Lej77](https://internals.rust-lang.org/u/Lej77)\
**Post date:** [May 4, 2022, 5:45pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/3 "2022-05-04T17:45:34Z")

</div>

`Pin` does have its [`Drop` guarantee](https://doc.rust-lang.org/std/pin/index.html#drop-guarantee) so it can be used in this way sometimes. Unfortunately the guarantee is not strong enough for APIs such as `thread::scope` since it only guarantees that data owned by the pinned pointer won't be invalidated before the pointer is dropped. It doesn't guarantee that borrowed data will remain valid until the pointer is dropped.

In the following code for example it would not be sound to allow other threads access to the `borrow` reference by assuming we could join in the `Drop` impl:

```rust
use core::marker::PhantomPinned;
use core::pin::Pin;

struct Guard<'a> {
    value: u32,
    borrow: &'a u32,
    pinned: PhantomPinned,
}
impl<'a> Guard<'a> {
    pub fn new(value: u32, borrow: &'a u32) -> Self {
        Self {
            value,
            borrow,
            pinned: PhantomPinned,
        }
    }
    pub fn start_background(self: Pin<&mut Self>) {
        // Send pinned values elsewhere (such as another thread)
        // We could send a pointer to `value` but not to `borrow`.
        println!("Start background")
    }
}
impl<'a> Drop for Guard<'a> {
    fn drop(&mut self) {
        // Drop guarantee ensures this is called before `Self` is deallocated.
        // It doesn't guarantee that this is called before `borrow` becomes invalidated since it is not owned by `Self`.
        println!("might never run if `Self` is forgotten");
    }
}

fn main() {
    let value = 2;
    let mut guard = Box::pin(Guard::new(4, &value));
    guard.as_mut().start_background();
    // Drop is never called (but we don't free the guard, so we don't break the "drop guarantee")
    core::mem::forget(guard);
}

fn alt_main() {
    let value = 2;
    let scope = async {
        // This could be in a separate async function:
        let guard = Guard::new(4, &value);
        tokio::pin!(guard);
        guard.start_background();
        // Yield:
        futures::future::pending::<()>().await;
    };
    let mut scope = Box::pin(scope);
    // Run to the yield point of the scope:
    futures::FutureExt::now_or_never(scope.as_mut());
    // Ensure drop impls after the yield point isn't run:
    core::mem::forget(scope);
}

```

[Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f14f047d340ef9652525ec6ea49b3673)

---

<div class="post-metadata">

**Author:** ![SkiFire13](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/skifire13/32/7579_2.png) [@SkiFire13](https://internals.rust-lang.org/u/SkiFire13)\
**Post date:** [May 4, 2022, 7:41pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/4 "2022-05-04T19:41:51Z")

</div>

I did propose a macro/stack-pinning approach in the scoped thread RFC ([link to the comment](https://github.com/rust-lang/rfcs/pull/3151#issuecomment-887512569)) but as @Lej77 said it was found to be unsound due to stack pinning not guaranteeing locals to be dropped in `async` blocks and functions.

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 4, 2022, 7:49pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/5 "2022-05-04T19:49:27Z")

</div>

i think introducing `with` block can maybe provide a stronger guarantee, so that the acquired resources/guards cannot transfer ownership, and thus the resources/guard cannot be called by forget or other function.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 4, 2022, 7:49pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/6 "2022-05-04T19:49:32Z")

</div>

Ah, and the same would apply to a `macro_rules!` based approach, as the extra behavior which using a closure gives you is that the enclosed code is no longer `async` and thus cannot `.await`, which it could if using drop glue emitted by a macro (effectively, pinned by hygiene)

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 4, 2022, 7:50pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/7 "2022-05-04T19:50:33Z")

</div>

What is the difference between

```rust
with let s = thread::scope_guard() {
    // ...
}

```

and

```rust
let s = &mut thread::scope_guard();
// ...

```

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 4, 2022, 7:54pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/8 "2022-05-04T19:54:01Z")

</div>

`with let x ... ` in semantic can limit operations to `x`. for example x cannot be moved, so you cannot transfer it to heap or called by `std::mem::forgot` because it requires argument by move.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 4, 2022, 8:04pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/9 "2022-05-04T20:04:06Z")

</div>

Note the `&mut` in my second example. Because the value is immediately put behind a reference, it cannot be used by-value either.

If you want to prevent `forget(mem::take(s))`, well... now every API which takes `&mut T` cannot be used with a `with let` value. (And then I ask, `let s = &expr;`.)

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 4, 2022, 8:08pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/10 "2022-05-04T20:08:09Z")

</div>

`with` block is different from the example. because `with` is a block, so you can not intersect the variable like the second example.

Sorry if this hypothetical syntax may be confusing.

In python, `with` block is like

```rust
with GetResource() as res:
    # do stuff with res

```

so I used `let` in hypothetical syntax to binding variable, but it is not a `let` statement variant. It is actually like:

```rust
with Variable_Initization {
  // do stuff
}

```

---

<div class="post-metadata">

**Author:** ![SkiFire13](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/skifire13/32/7579_2.png) [@SkiFire13](https://internals.rust-lang.org/u/SkiFire13)\
**Post date:** [May 4, 2022, 8:18pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/11 "2022-05-04T20:18:10Z")

</div>

How do you uphold those guarantees though? I don't see how that would be possible while allowing `.await` inside `with` blocks. The problem is that while you can not `mem::forget` the resource created in the `with` block you can `mem::forget` the whole `Future` that contains it, thus forgetting the resource in the process.

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 4, 2022, 8:23pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/12 "2022-05-04T20:23:12Z")

</div>

Maybe forbid await inside `with`? Otherwise, I think `std::thread::scope` api may suffer from this problem too.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 4, 2022, 8:38pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/13 "2022-05-04T20:38:30Z")

</div>

> [@xjkdev](#):
>
> you can not intersect the variable

I think that's a typo; I don't know what it means to "intersect" a binding.

> [@xjkdev](#):
>
> Maybe forbid await inside `with` ? Otherwise, I think `std::thread::scope` api may suffer from this problem too.

Then at that point, what's the advantage over introducing a closure? A closure naturally restricts access to `.await`, as well as any other future control flow which could potentially cause issues.

---

<div class="post-metadata">

**Author:** ![y86-dev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/y86-dev/32/9867_2.png) [@y86-dev](https://internals.rust-lang.org/u/y86-dev)\
**Post date:** [May 4, 2022, 10:25pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/14 "2022-05-04T22:25:18Z")

</div>

Could you not wrap any object guarded by `with` with `ManuallyDrop`, defeating the `Drop` guarantee introduced by `with`, of course you could detect such usage and emit an error, but then you would also need to traverse newtypes.

I do not really see a use-case for this. When you really need to defer some teardown, creating a zero size struct/closure wrapper with the teardown in the `Drop` impl is a good way to handle this. It is already used in many places and provides everything that `with` would also provide.

Also when do you come across the situation where you want something to be dropped, but use `mem::forget` anyway? I rarely use `mem::forget` (especially not when i am handling resources that need dropping), so i am curious for your use case.

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 5, 2022, 10:11am UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/15 "2022-05-05T10:11:44Z")

</div>

There are currently a lot resources and guards that are not using closure. so, they may be forgotten directly or indirectly in previous examples. Introducing `with` block, can help user to mark those they do not want to forget.

---

<div class="post-metadata">

**Author:** ![xjkdev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xjkdev/32/7359_2.png) [@xjkdev](https://internals.rust-lang.org/u/xjkdev)\
**Post date:** [May 5, 2022, 10:23am UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/16 "2022-05-05T10:23:33Z")

</div>

Like previous examples, forgetting and memory leaks may be indirectly. `with` can provide closure-like guards to all resources.

However, the closure wrappers can be difficult to look when managing multiple resources, and hard to analyze for compiler. for example, if you have three guards, it may look like:

```rust
guard_a(|x| {
  guard_b(|y| {
    guard_c(|z| {
       // do real stuff here
    }
  }
}

```

but with `with` block, guards can be initialized in one line.

```rust
with let x=guard_a, let y=guard_b, let z=guard_c {
  // do stuff here
}

```

That's why i think `with` can be better than closures.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 5, 2022, 12:36pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/17 "2022-05-05T12:36:30Z")

</div>

> [@xjkdev](#):
>
> Introducing `with` block, can help user to mark those they do not want to forget.

So what is meaningfully different with using

```rust
let x = &mut guard_a();
// which is just a shorter version of
let mut x = guard_a();
let x = &mut x;

```

This also ensures that the value is only used by reference.

---

<div class="post-metadata">

**Author:** ![y86-dev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/y86-dev/32/9867_2.png) [@y86-dev](https://internals.rust-lang.org/u/y86-dev)\
**Post date:** [May 5, 2022, 12:36pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/18 "2022-05-05T12:36:46Z")

</div>

> [@xjkdev](#):
>
> Like previous examples, forgetting and memory leaks may be indirectly. `with` can provide closure-like guards to all resources.

I could not find such an example, could you maybe show some running rust code that "indirectly" `mem::forget`s a value? I have never come across such a situation, so i find it really hard to image how it happens.

> [@xjkdev](#):
>
> However, the closure wrappers can be difficult to look when managing multiple resources, and hard to analyze for compiler. for example, if you have three guards, it may look like:
> 
> ```rust
> guard_a(|x| {
> guard_b(|y| {
> guard_c(|z| {
> // do real stuff here
> }
> }
> }
> 
> ```

Not if you use a macro that simplifies it:

```rust
macro_rules! with {
    (let $var:ident = $guard:ident => { $($inner:tt)* }) => {
        $guard(|$var| {
            $($inner)*
        })
    };
    (let $var:ident = $guard:ident , $($tail:tt)*) => {
        $guard(|$var| {
            with!($($tail)*);
        })
    }
}
#[derive(Debug)]
struct GuardA;
#[derive(Debug)]
struct GuardB;
#[derive(Debug)]
struct GuardC;

fn guard_a(_: impl FnOnce(&mut GuardA)) {
    todo!()
}
fn guard_b(_: impl FnOnce(&mut GuardB)) {
    todo!()
}
fn guard_c(_: impl FnOnce(&mut GuardC)) {
    todo!()
}

fn main() {
    with! { let x = guard_a, let y = guard_b, let z = guard_c => {
            // do stuff here
            println!("{x:?}, {y:?}, {z:?}");
        }
    }
}

```

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [May 5, 2022, 3:44pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/19 "2022-05-05T15:44:43Z")

</div>

In Python, the with statement means exactly the same as creating a local and dropping it at the end of the block scope in Rust, it's just (superficially) more complicated, and the reason it's necessary is that the language doesn't guarantee deterministic destruction like Rust does universally.

I fail to see how adding different syntax for the exact same purpose would help — there have been _many_ such "this syntax would be cute/I like it better/it is familiar to Language X users" proposals in the past, but this argument doesn't nearly reach the threshold for changing the core language. If you like the Python syntax better, feel free to use Python, just do not expect Rust to be changed solely based on what other languages do.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [May 5, 2022, 5:29pm UTC](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577/20 "2022-05-05T17:29:52Z")

</div>

> [@H2CO3](#):
>
> I fail to see how adding different syntax for the exact same purpose would help

I feel this is a bit harsh; @xjkdev has multiple times expressed that they believe that `with` scoping would be semantically different from a plain `let` binding. Whether the semantic difference is real or meaningful is a separate question, but @xjkdev is _not_ proposing different syntax for the same purpose, they are proposing something they believe is an extension of what can be semantically expressed.

(As my posts show, I disagree with this understanding, and believe that taking an immediate `&mut` reference achieves the exact benefit they're after. But the point stands that they _aren't_ just proposing sugar.)

[Next page](https://internals.rust-lang.org/t/idea-with-block-to-ensure-definite-resource-release/16577.md?page=2)