Hello, the reason we can't move out of mutable references and then borrow checker controlling that it was moved back is unwinding. But when panic=abort, there can't be any unwinding. Can we then make it legal to move out of mutable references?
I thought so too, and got about 70% of the way through writing an unsafe kludge to let my panic handler steal the global LineWriter wrapping stderr from my main program, but then I realized I was wrong. It's still unsound (not just unsafe) to steal mutable access to something, even when there's no unwinding and no threads, because of reentrancy effects. The panic handler could have interrupted execution of the guts of the LineWriter and therefore cannot safely claim it even though the main program will never use it again.
I'm not sure this is exactly what you have in mind, though. Can you give a concrete example of the code you'd like to be able to write and presently can't?
6 Likes
I was thinking about non-aborting equivalent of this popular crate take_mut - Rust. It gives you T and on panic catch_unwind your closure, aborting the process
Implement mem::replace using direct, safe code:
fn replace(dest: &mut String, src: String) -> String {
let result = *dest;
*dest = src;
result
}
Mentioned by Niko in this blog post. The missing tweet was something like "Chanting in the distance: panic=abort having semantic impact" IIRC.
I also don't feel like config option changing semantics is "clean", but on the other hand... Entire ecosystems (embedded for example, or wasm frontends, etc) paying costs like that and more for the thing we are not even using? It is sad...
2 Likes
I have 2 notes on argument against this in that article:
The idea of aborting the process is that, unlike unwinding, we are guaranteeing that there are no more possible observers for that hole in memory. Interestingly, in writing this article, I realized that aborting the process does not compose with some other unsafe abstractions you might want. Imagine, for example, that you had memory mapped a file on disk and were supplying an &mut reference into that file to safe code. Or, perhaps you were using shared memory between two processes, and had some kind of locked object in there – after locking, you might obtain an &mut into the memory of that object. Put another way, if the take_mut crate is safe, that means that an &mut can never point to memory not ultimately “owned” by the current process. I am not sure if that’s a good decision for us to make – though perhaps the real answer is that we need to permit unsafe crates to be a bit more declarative about the conditions they require from other crates, as I talk a bit about in this older blog post on observational equivalence.
First is, to my knowledge, it is (probably) already unsound for other "entity" to read/write memory of Rust code. It is not only touching those theoretical scenarios like mmap, but a highly practical one: you can't touch memory of wasm from host function called back by rust running inside. At least wasmi and wasmtime are not allowing it by mutaby borrowing access to the memory. I saw some folks implemented logging with unsafe, but it is not passing miri. (I did it by passing values on stack, not sure what wasm_bindgen is doing, they execute from browser so no one is probably preventing them from reading).
Secondly, it states
and had some kind of locked object in there – after locking, you might obtain an &mut into the memory of that object.
When I have &mut T and something is not aliasing due to that locking, then when I move out of reference it is still locked. If process aborts, nobody unlocked anything. Won't they touch memory before unlock?
It is up to us to define what next - for example, we can say that abort causes those uninit bytes to freeze.
And, additionally, I don't think llvm terms can really apply between processes - those are mmap'ed files. If you want to use mmap today, inside 1 process, to, for example, create a circular buffer, then you are out of luck and need to use pointers and volatiles. I can't imagine &mut T into mmap file...
To sum up: this is not just "a hole in memory", but "a mutaby borrowed hole in memory".
1 Like
Here's some idea: some sort of language construct that guaranteed that certain piece of code doesn't panic. Then, inside that you could temporarily move out of mutable references, without leaving a sentinel value.
Something like this:
fn replace(dest: &mut String, src: String) -> String {
nopanic {
let result = *dest;
*dest = src;
}
result
}
3 Likes
I think it would be really cool to have this nopanic guarantee, not just for this reason.
What's the most recent discussion on this?
Does rust provide any panicking guarantees about eg. basic arithmetic? I'm curious just how much you actually can portably guarantee on the hardware out there.
You can just be very conservative: each operation that might potentially panic can't be called in a nopanic { } block at all. Gradually the allowed operations could be expanded, just like const { .. } works today.
The real trouble is that making a non-panicking function panic is a semver-compatible change, so inferring whether an existing function panics would be too brittle (just imagine that after a minor upgrade to a crate, your code could randomly fail to compile). So, for this feature to work right we need a language-level nopanic fn notation on functions, again just like const fn, to declare a guarantee that this function will continue to not panic as the crate evolves (adding nopanic to a function is not a breaking change, but removing it is). Then, only a nopanic fn could be called inside a nopanic { .. } block, and it's up to each crate (including the stdlib) to properly annotate their APIs.
To avoid the ecosystem churn, the best time to introduce such nopanic fn notation was before Rust 1.0; the second best time is today.
2 Likes
This is already discussed in context of effects. For example, const traits are being rewritten with them.
Note, that this topic is about unwinding, not panics.
You can just still use replace_with_or_abort in replace_with - Rust and its panic handling will optimize out with panic=abort, no?
I personally just have custom aborting function (it is not trivial to abort in no_std), but why if borrow checker can do the work? Those abstractions are really unpleasant to work with tbh.
Is it? panic-while-unwinding is an abort, so you can always do that, even in no_std https://rust.godbolt.org/z/cGc7rn4hs
(That's what replace_with uses to avoid needing catch_unwind the way take_mut does.)
Sometimes you don't have unwinding as an option, it is directly not supported at all.
That's part of the reason why I say this. Sometimes you just can't unwind, there is not an option to unwind. Code will not compile without panic="abort" Yet still you pay for it.
What about a nounwind block that generates an abort landing pad?
I feel that changes to "normal" borrow checker behavior should be locally visible.
The pattern to turn unwinds into aborts is even available as an unstable API:
I'm slowly chipping away at that. There's core::intrinsics::abort, and I'm slowly working on exposing abort in core, progress just got held up by life and git randomly deciding it was going to fail every fetch for some reason (And then eventually randomly fixing itself.)
2 Likes
I think I want to write a pre-rfc for this. How do you think libs should handle it? If they are moving out of references then if user has panic=unwind then it would be really bad ux to give "cannot move out" error from within that lib.
So libs should be able to specify it in their Cargo.toml or like that, ideally with ability to feature gate it.
If just in lib.rs, then maybe like
#![cfg_attr(not(feature="unwind"), /* declare that panics cannot unwind*/]
#[cfg(feature = "unwind")]
fn swap<T>(a: &mut T, b: &mut T) { ... }
And add unwind as default, so that power users will opt out of it, if they have panic=abort. That way rustc will bail the library if it uses something incompatible with unwind before declaring it.
Or maybe just #[cfg(unwind)], but that way library author is responsible to not put incompatible code out of cfg.
#[cfg(unwind)]
fn swap<T>(a: &mut T, b: &mut T) { ... }
Additional thoughts:
- It will not cause an ecosystem split, as it be mainly used in ecosystems that are already separated from "main stream" (examples: embedded, wasm) or feature gated. E.g: if you want this feature, disable unwinding - seems really reasonable to me.
- I personally think that even for web servers that want to handle panics gracefully, they are already relying on auto scaling from orchestrators like kubernetes for zero downtime, a single instance aborting must be expected.
- Multithreading is really brittle in terms of errors in threads, you can't soundly terminate any, while ipc is not that bad, especially with crates like this crates.io: Rust Package Registry .
Alternatively, maybe effects can play some role in here? As far as I can see, they are not really going anywhere except for const stuff, and even async folks decided to go with AsyncFn instead of async Fn, a small step in the opposite direction. And even if eventually there will be an effect like this, migration should be easy, through the edition, so that "hack" (with a bunch of benefits for nearly everyone) will not make it harder for a "proper" solution like an effect - it will do the opposite, show case their benefit to Rust.
Idea: can we allow threads to abort on panic instead of unwind or abort the process, while absolutely isolating it? From the point of any code it should be observable like it has loop {}, in other words:
If thread captured 'static data, the we will end up with leaked mutexes, as there is no one to unlock them.
If thread captured non static data, then currently it is only possible with scope. Scope can just never end, assuring that data remain borrowed. Thread will no harm to & data, while no one will be able to observe bad &mut data.
Paradoxically, even if it is "leaking" the whole thing, it is not incompatible with unleakable or linear types, as Rust program can assume that thread just went in loop {} (or got unlucky with scheduling) , while actually being terminated by os. The rest of the program can move on but with some resources being borrowed "until that loop ends" - by thread capturing 'static, by scope never returning, by linear type (JoinHandle)
"guarding" the thread and keeping borrowing resources until some message from the thread, that will never come.