Pre-RFC: `Stdio::dup_stdout()`

The std::process::Stdio type is used in std::process::Command to allow you to set the stdin, stdout, and stderr streams in the child process being spawned. However, the Stdio type takes ownership over any method of passing it an existing file descriptor, which makes it hard to use if you want to redirect the child process's stdout and stderr streams into the same place. Thus, I propose adding an option to explicitly make stderr duplicate the file descriptor from stdout and use that.

This will be an error to provide into Command::stdin and Command::stdout (caught when you try to spawn), but when provided to Command::stderr, it handles the stdout stream first, then duplicates the file descriptor onto stderr.

Why not open the same target twice? Having independent file descriptions is likely less troublesome than duping a file descriptor.

If I want to save stdout and stderr to the same file, I could go through a big hassle with OpenOptions to create or truncate a file for one stream, and then have the second stream append to the file. This results in significantly different code for opening the two files (creating vs. appending, and maybe different methods of error handling cases, since you expect different things), and risks getting weird states if some other program messes with the filesystem between the two calls (maybe it moves a containing directory and makes a replacement, so the path now refers to a different file). And then, when someone goes to read the code, they have to figure out that the two methods are the same, up to the differences of one making a new file and then the other appending to the same file.

Alternatively, duping the file descriptor doesn't have to worry about any edge cases caused by some other process concurrently messing with the disk, and makes it immediately obvious to anyone reading the code that the stderr goes to the same place as the stdout.

And, if someone comes along later to change where the output goes, they just have to change how they set the stdout stream and stderr will change along with it for free, instead of having to figure out how to make a new copy that appends to the same place (which may be different depending on where stdout comes from).

Well, nevermind. I just ran 0/1/2 in a shell-spawned rust process through kcmp and it looks like they're the same file descriptions anyway. So yeah, dup should be fine. You can get that today by using File::try_clone or BorrowedFd::try_clone_to_owned before constructing the Stdio types.

That workaround doesn't work if you want to do this with Stdio::piped(), which afaict doesn't make the file descriptor until you spawn the child process. And I'm sure you could figure out how to work around that with some other clever thing, but IMO this is something that comes up often enough that you shouldn't have to figure out how to be clever about it.

You can call piped any number of times, there's no need to dup it. And no, I wouldn't call "call this method twice" a clever workaround.

But if I call it multiple times, it makes multiple pipes. If I only want one pipe, that means I have to figure out something clever to combine the streams (see above post about being clever).

Oh yeah, but those aren't actual OS pipes that can be passed forward arbitrarily. They're "intent to connect 1:1" placeholders so they can be fed into the two separate Vec<u8> in process::Output.

If you want actual OS pipes then std doesn't have a public API for that and you'll have to use libc::pipe or nix::unistd::pipe to create an anonymous fifo and then pass those as you would pass File.

1 Like

Is dup_stdout something that can be implemented portably? At least in Unices and Windows

There are multiple libraries in the ecosystem for portably creating pipes. We might want to evaluate whether we should have support for that in std.

3 Likes

Generalizing, it can also be useful to allow a child process to borrow (in the sense that Rust uses that term) access to a file that remains open in the parent. The most obvious example I can think of, is when you've created an anonymous temporary file and you want the child to write its output to that file. (Maybe you can't use a pipe because the child requires its output file to be seekable.)

Allowing Stdio to be created from &mut File and from BorrowedFd / BorrowedHandle, with the semantic that it uses dup2 to give the child process access to the open file as the appropriate standard stream, and the parent is locked out of the file until the child terminates, would make the above possible pretty neatly:

let mut tmpfile: File = tempfile::tempfile().unwrap();
Command::new("child-program")
    .stdin(Stdio::null())
    .stdout(&mut tmpfile)
    .spawn().unwrap().wait().unwrap().exit_ok().unwrap();

tmpfile.rewind().unwrap();
// read from tmpfile...

You don't need any new API for that, try_clone + Stdio::from(file) works for that purpose.

and the parent is locked out of the file until the child terminates,

You wouldn't get this property anyway because Command doesn't have a generic lifetime.

You can't get locked out of the file until the child terminates because there's no generic lifetimes on Command/Child types to enfore borrow check rules (unless we make a new backwards-incompatible interface in addition to the current one, but that seems very unlikely to me, and would need something stronger than this to motivate it).

You could allow constructing a Stdio from a BorrowedFd with having to promise that you don't close the file descriptor until after spawning the child. I think this could be made safe, since there's no risk of UB if the file descriptor is closed (either you get an error trying to use the file descriptor, or worst-case scenario the child process gets some other, newly-opened file descriptor instead and you get behavior which is nonlocally weird, and definitely not nice to have, but still fully defined).

dup2 is a standard syscall, so it should be supported on any Unix. I don't know anything about file handling on Windows, but I'd assume they have something to support it (and if not, this could go in a unix-specific extension trait, we've got plenty of those running around).

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.