Hey all! The standard library provides a number of synchronization primitives for all Rust programs to use in a cross-platform fashion. Ideally the standard library also provides sound implementations of these primitives so they can’t stop working at runtime!
Currently the system primitives
ReentrantMutex (internal to libstd) all wrap underlying primitives for each os (pthreads on non-Window, corresponding objects on Windows). Unfortunately, though, using the OS primitives brings quite a few caveats:
- First off, once the primitive is used its memory address cannot be changed. This requires our safe wrappers to use
Boxto contain the primitives (as Rust values can change memory addresses through moves).
- Some primitives like
Condvarhave restrictions such as they can only ever be used with at most one mutex. It’s undefined behavior with pthreads (I think) to use more than one mutex with a cond var (over its entire lifespan, not necessarily at the same time).
- Reentrantly acquiring a
Mutex(aka locking it twice on the same thread) is undefined behavior by default in posix, so we have to handle this with explicit initialization
And that’s just the ones we’ve managed to fix! There’s a number of open issues that don’t currently have proof-of-concept programs showing unsoundness, but we’re still causing undefined behavior in the “spec says it’s undefined” sense:
- RwLock on posix is also undefined behavior to acquire reentrantly, or at least for the write thread.
- Mutex rwlocks are also undefined behavior to access reentrantly
Overall this makes the current situation in the standard library pretty unfortunate! We have already gone to great lengths to try to work around the OS primitives, but we’re still not all the way there in terms of allowing 100% of safe Rust to leverage the primitives without undefined behavior. We think that this represents the set of guarantes we need to uphold (basically fixing reentrant locking of rwlock on all platforms), but there could be more future gotchas as well!
We had a short discussion about these issues at the recent libs triage meeting, and we’d like to send out a call for help and suggestions of how to fix these! Correctness here is absolutely paramount, and the secondary concern for these primitives is speed. There seems to be two broad methods that we could go about solving these issues:
- Continue to hack around the OS primitives in the standard library. It’s unclear what the performance hit would be for fixing the rwlock reentrancy, and it’s also somewhat unclear what the solution would look like! (there aren’t any proposals yet).
- Alternatively we could take it upon ourselves in the standard library to reimplement these primitives without basing them on the system implementations. The most prominent implementation for doing this
It may seem obvious that "just use
parking_lot" is the solution here but it’s unfortunately more nuanced than that (some more discussion can be found on an old RFC as well). The libs team is not ready to simply move
parking_lot into the standard library and call it a day, but we would prefer to move more carefully and deliberately to measure performance, ensure stability guarantees can be upheld, and ensure that platform compatibility is maintained. Additionally we’d ideally like to see a solution that continues to rely on system primitives to have a “bottom line” implementation to start from and evaluate an alternative strategy like
Would others be willing to help us out with these issues? @Amanieu or other
parking_lot folks, would y’all be willing to help work with us and figure out what it would look like for
parking_lot to be in the standard library? Do others have ideas on how to fix these issues?
Help is always greatly appreciated here!