(NOT A CONTRIBUTION)
tokio has a concept of a cooperative task scheduling budget, by which its futures keep track of how much work they've done, and cooperatively yield if they've done too much work. This way, a single task which has a lot of work to do can't hog the thread, blocking other tasks from also completing work. This improves tail latencies. Read more here: Reducing tail latencies with automatic cooperative task yielding | Tokio - An asynchronous Rust runtime
It would be interesting if Rust's task system could support this natively, so different libraries could use the same cooperative scheduling system without depending on a specific third party runtime that sets a thread local. To do this, the budget would need to be communicated via the task Context
.
The tokio system only has two APIs that are used by the various synchronization and IO primitives:
-
poll_proceeding
, which attempts to consume some budget and returnsPending
if we're out of budget, or an object calledRestoreOnPending
if we aren't not. -
RestoreOnPending::made_progress
which indicates that this task has made progress.
(Note: the importance of tracking if we've made progress is that RestoreOnPending
gives budget back if this future didn't make progress, so that tasks that are not actually making progress don't run out of budget while trying to complete.)
There are other APIs in tokio relating to budget, but they are all essentially internal to tokio's executor. To write a new concurrency or IO primitive, these are the only two you need.
I'm imagining a new submodule of task
, called coop
. In the coop module, there is a type called Cooperation
, which represents an attempt to cooperatively share the task budget. This type is similar to Waker
- it has a word size data component and vtable. The vtable for now would have only two methods: make_progress
, to indicate that this future has made progress, and drop
, for when it drops. In the future, this vtable could gain other optional methods. There would also be a default constructor for Cooperation
, which does nothing. Cooperation
does not implement Send
or Sync
.
mod coop {
pub struct Cooperation {
data: *mut (),
vtable: &'static CooperationVTable,
}
impl Cooperation {
// a cooperation in an executor with no budget constraints
pub fn unconstrained() -> Cooperation { ... }
pub fn made_progress(&mut self) { ... }
}
impl Drop for Cooperation {
fn drop(&mut self) { ... }
}
impl !Send for Cooperation { }
impl !Sync for Cooperation { }
pub struct CooperationVTable {
made_progress: fn(*mut ()),
drop: fn(*mut ()),
}
// plus a bunch of constructors similar to Waker's
}
Context would gain a constructor that also takes a virtual function that constructs the Cooperation
. Because Context
is not Send
or Sync
, you can be sure that this function will only be called on the same thread (so e.g. it could access a thread local). This function would return a Poll<Cooperation>
. Context would gain a method which calls this if it is set, and returns the null cooperation if it is not (indicating a runtime that doesn't have a notion of task scheduling budgets):
impl Context<'_> {
pub fn poll_cooperate(&mut self) -> Poll<Cooperation> { ... }
}
// hypothetical API also used in LocalWaker proposals
impl ContextBuilder {
fn set_poll_cooperate(
&mut self,
poll_cooperate: fn() -> Poll<Cooperation>,
) { ... }
}
Objects that want to participate in a cooperative yielding system like tokios would, when they are polled, first call ready!(cx.poll_cooperate())
to get a Cooperation
. When the poll method makes progress, they would note made_progress
on the Cooperation
, and when it returns they would drop the Cooperation
.
Thoughts appreciated.