I don't know how we could go about doing something like this, but it would be nice if there was a lint for blocking functions in async code. Occasionally I forget I'm an async function and I do blocking calls. I imagine that in single threaded aplications something like this could be really useful. Has it ever happened to you?
1 Like
toc
October 26, 2021, 6:03pm
2
Seems like there's some traction.
opened 01:31AM - 12 Nov 20 UTC
In order to write reliable and performant async code today, the user needs to be… aware of which functions are blocking, and either:
- Find an async alternative
- Schedule the blocking operation on a separate thread pool (supported by some executors)
Determining if a function may block is non-trivial: there are no compiler errors, warnings or lints, and blocking functions are not isolated to special crates but rather unpredictably interspersed with non-blocking, async-safe synchronous code. As an example, most of `std` can be used in async code, but much of `std::fs` and `std::net` cannot. To make matters worse, this failure mode is notoriously hard to detect: it often compiles and runs fine when the executor is under a small load (such as in unit tests), but can cause severe application-wide bottlenecks when load is increased (such as in production).
For the time being, we tell our users that if a sync function uses IO, IPC, timers or synchronization it may block, but such advice adds mental overhead and is prone to human error. I believe it's feasible to find an automated solution to this problem, and that such a solution delivers tangible value.
## Proposed goal
- Offer a way to declare that a piece of code is blocking.
- Detect accidental use of blocking functions in async contexts.
- Display actionable advice for how to mitigate the problem.
- Offer an override so that users and library authors who "know what they're doing" can suppress detection.
## Possible solutions
I am not qualified to say, and I would like your thoughts! A couple of possibilities:
- A new annotation, perhaps `#[may_block]` that can be applied on a per-function level.
- An auto-trait or other type system integration (although this would be much more invasive).
## Challenge 1: Blocking is an ambiguous term
Typically, blocking is either the result of a blocking syscall or an expensive compute operation. This leaves some ambiguous cases:
- Expensive compute is a gray area. Factoring prime numbers is probably blocking, but where's the line exactly? Fortunately, very few (if any) functions in `std` involve expensive computation by any reasonable definition.
- A method like `std::sync::Mutex::lock` is blocking only under certain circumstances. The `must_not_await` lint is aimed at preventing those circumstances (instead of discouraging its use altogether).
- `TcpStream::write` blocks by default, but can be overridden to not block using `TcpStream::set_nonblocking`.
- The `println!` and `eprintln!` macros may technically block. Since they lack async-equivalents and rarely block in practice, they are widely used and relatively harmless.
- Work-stealing executors may have a higher (but not an infinite) tolerance for sporadic blocking work.
Ambiguity aside, there are many clear-cut cases (e.g. `std::thread::sleep`, `std::fs::File::write` and so on) which can benefit from a path forward without the need for bike-shedding.
## Challenge 2: Transitivity
If `fn foo() { bar() }` and `bar()` is blocking, `foo()` is also blocking. In other words, blocking is transitive. If we can apply the annotation transitively, more cases can be detected. OTOH, this can be punted for later.
## Challenge 3: Traits
When dynamic dispatch is used, the concrete method is erased. Should annotations be applied to trait method declaration or in the implementation?
## Challenge 4: What is an "async context"?
The detection should only apply in "async contexts". Does the compiler already have a strict definition for that? Examples from the top of my head:
- Directly in an async block or function.
- Indirectly in an async block or function, e.g. in a closure.
- Inside the `poll` method of a custom future impl, or the `poll_fn` macro.
## Background reading
- [Comment in the must_not_await issue](https://github.com/rust-lang/wg-async-foundations/pull/16#issuecomment-661457872).
- [Thread on rust-internals](https://internals.rust-lang.org/t/warning-when-calling-a-blocking-function-in-an-async-context/11440).
opened 12:42PM - 13 Aug 19 UTC
A-lint
T-async-await
<!--
Hi there! Whether you've come to make a suggestion for a new lint, an impr… ovement to an existing lint or to report a bug or a false positive in Clippy, you've come to the right place.
For bug reports and false positives, please include the output of `cargo clippy -V` in the report.
Thank you for using Clippy!
Write your comment below this line: -->
With `async/await` coming up, we will probably see an uptake of such code:
```
async hello() {
println!("hello").
}
```
Users will most likely not be aware that the call to `println` are _blocking_. The same goes to any use of the stdlib io apis, most famously `to_socket_addr()` calls.
I think there is a chance for clippy to at least find obvious cases of this, for a list of known functions.
2 Likes
system
January 24, 2022, 6:08pm
4
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.