println! will panic when stdout is closed. There's proposed #[unix_sigpipe = "sig_dfl"] to address this, but it elevates panics to killing the process, which for my use-cases only makes things worse. I'm missing a third option that ignores the error (sig_ign ignores the signal, but not the write error).
println! panic is a recurring annoyance for me in server development, because it tends to happen during server shutdown and causes noise in crash reports. My servers need to support zero-downtime restarts, so they're not supposed to be suddenly killed on restart whenever that can be avoided. They also have error reporting like Sentry that makes a big fuss about every panic (usually for a good reason). But there are conditions where a supervisor of the server process closes it stdout/stderr before the server quits, and then any print during shutdown can cause a panic. Some server components can become chatty during shutdown when things they depend on become unavailable, and use of println! is quite common, and replacing with something like let _ = write!(stdout(), "") is a whack-a-mole.
So I'm wondering, could there be a way to defuse println!?
Maybe as a different flavor of #[unix_sigpipe = "sig_dfl"] that means ignoring both the signal and the error in (e)println!, or maybe some libstd function, that I could call at the start of shutdown sequence, that uses its stdout capture mechanism to redirect println! to /dev/null or give it an error-swallowing wrapper on stdout.
I've thought for a while that closing std::io::std{in,out,err} should actually replace them with a handle to /dev/null, at the level of OS file descriptors. Thus, your "libstd function, that I could call at the start of shutdown sequence" would be io::stdout().close().
rustc has needed this so much it has it's own safe_print macro that avoids ICEing when there's an error (it still raises a fatal error but that's different to bare panicking).
Isn't best practice for servers to use the tracing or log ecosystems of crates rather than println? Maybe that is a better way to deal with the problem for your use case. Then your logging/tracing handling crate just needs to do its printing correctly.
myrrlyn wrote the calm_io crate a while back which provides stdoutln! as an alternative to println! which returns the io::Result instead of unwrapping it.
I believe clippy and/or cargo-deny are capable of linting on println! usage to assist in playing whack-a-mole.
Hmm. How often does one want this but not want to wrap stdout in a BufWriter? I have been working on a CLI program lately that does a lot of output to stdout. Its main looks something like
fn main() -> ExitCode {
let args = parse_cmdline(); // doesn't return on error or --help
let mut stdout = io::stdout().lock();
// ... wrap stdout in a BufWriter if it's not a tty ...
if let Err(e) = run(stdout, args) {
eprintln!("{}", e);
return ExitCode::FAILURE;
}
ExitCode::SUCCESS
}
and then everything that wants to print to stdout uses writeln!(stdout, ...)? and it works out nicely.
For stderr it makes more sense, up to a point: if writing to stderr doesn't work, where do you report that error?? In the above snippet I'd be happy for the eprintln! to ignore errors, since the program is about to exit unsuccessfully anyway. In a program like rustc, where stderr might get lots of output in mid-execution, I think I'd go with passing down a handle and using writeln! and discarding failures.
There's proposed #[unix_sigpipe = "sig_dfl"] to address this, but it elevates panics to killing the process
I would like to clarify how #[unix_sigpipe = "sig_dfl"] works and what it does, with the hope that it will be helpful. I would not say that it "elevates panics to killing the process".
A write syscall to the fd of a closed pipe will trigger the SIGPIPE signal. For crate type bin, Rust by default sets the SIGPIPE handler to SIG_IGN before fn main() is invoked. This causes SIGPIPE to be ignored, and the write syscall will return EPIPE, which will be converted to a std::io::ErrorKind::BrokenPipe. The println() implementation will unwrap() this error, causing a panic.
With #[unix_sigpipe = "sig_dfl"], Rust will set the SIGPIPE handler to SIG_DFL before fn main() is invoked. Now when a write syscall to the fd of a closed pipe triggers SIGPIPE, the process will be killed before write returns (which is the default behavior of SIGPIPE). A panic will never occur, because the process is killed before there is any chance to trigger a panic. Note in particular that this means that destructors will not be run (which explicitly is not UB in Rust).
So I'm wondering, could there be a way to defuse println!?
Maybe, but I don't think #[unix_sigpipe = "..."] will be useful for it.
In my opinion, println! is not intended for production systems or tools, but for use cases that are closer to hello world or debugging that don't care about absolute correctness.
SIGPIPE is a complex question with many possible behaviors, and I think that for more "serious" applications that care about handling it correctly, Rusts SIG_IGN is the correct solution because it allows the application to handle it however they want in different places. Dependong on what they're printing and the importance of that print, behavior might be different. A CLI that prints output without doing any side effects will probably want to exit, so it can define a wrapper around write!(stdout()) that handles ErrorKind::BrokenPipe and exits (or it could use the unix_sigpipe attribute, but I don't particularly like that attribute, I think writing a custom wrapper is nicer as it makes it more explicit when we may exit).
For server logs, just ignoring the error makes a lot of sense. If a a logging library like tracing or log is used, this can be centralized in one place.
I don't think we should just ignore errors by default. If the user wanted to write something somewhere, but writing it failed, that's an error condition in general. Whether it actually is an error that should be displayed, or whether it should just be ignored should be up to the programmer.
As for whack-a-mole-ing out println!, Clippy has lints for this that can be used to reliably avoid println! (which in my experience isn't a big problem for server software when a nice logging framework like tracing is used, which is really nice to use, but I haven't written that much server code in Rust): Clippy Lints
Just to note, there is a meaningful difference here. On SIGPIPE, a unix process could:
Continue on screaming into the /dev/null void.
Exit gracefully, claiming success with exit(0).
Exit gracefully, claiming error with exit(1..=255).
Exit nongracefully, attempting to complain to stderr (e.g. panic!).
Exit abruptly with SIGPIPE.
That last one IIUC cannot be replicated without the #[unix_sigpipe] attribute, and looks different to the parent process, as it causes the process to report an exit code of 256 + SIGPIPE, which is not possible except for process killed by SIGPIPE. This difference can have an impact on how shells respond to process termination.
There is a way, but it's a bit of a pain: at the point where you want to turn the EPIPE error back into a fatal SIGPIPE, join all other threads that are still running, unblock SIGPIPE, reset its disposition to SIG_DFL, and then raise(SIGPIPE). I can't think of any program that actually bothers for SIGPIPE, but I have seen guides that encourage you to do the equivalent at the end of clean-up-before-exiting SIGINT (^C) handlers, so the technique is out there in people's minds.
EDIT: It occurred to me immediately after posting that this could almost be done generically in std-for-Unix by adding a from_signal constructor to ExitCode. (It would have to be fallible, accepting only those signal numbers whose default action is to kill the process. There is no way to make a signal whose default action is "do nothing", like SIGCHLD or SIGWINCH, kill the process.)
Returning that variant of ExitCode from main, or calling exit_process on it, would carry out the above sequence ... except there's no good way to do the "join all other threads" part, and without that you have no guarantee that you're not racing with some other thread that wants to block the signal in question or change its disposition.
Interesting, but hardly practical, specifically about joining all other threads: Consider for example a thread pool with rayon, I don't think there is even a way to stop the default thread pool in that case (this is relevant to a project I'm currently writing the early parts of, though the use of rayon parallel iteration is set in stone at this point).
Why do you need to join other threads for the signal handling changes in this case though?
Why do you need to join other threads for the signal handling changes in this case though?
The signal handling settings are process-global. Another thread could revert the changes to the mask and/or disposition for SIGPIPE, after the first thread's changes but before the raise(), causing the raise() to be ineffective.
Right, makes sense. And something std or a library has to worry about (but can't do much about).
But I don't know of many (any?) rust library that messes with signals internally (unless that is their purpose, eg. signal/sigaction in libc, or other crates dealing specifically with signal handling). Main reason of course being that signals are seldom the best way to deal with anything if you have another option. So for a typical application developer it is likely not a real problem.