How long does it take if you use poll with a zero timeout and no events to test a bunch of fds all at once? That is what libstd uses to check if fd 0, 1 and 2 are open or not at startup.
That's much faster (on my machine, poll calls are limited to 1024 fds per call, idk how common that limit is):
#include <stdio.h>
#include <stdlib.h>
#include <poll.h>
int main() {
int fd_per_poll = 1024;
struct pollfd* fds = (struct pollfd*) malloc(fd_per_poll * sizeof(struct pollfd));
for (int min_fd=0; min_fd < 268435456; min_fd += fd_per_poll) {
for (int fd = 0; fd < fd_per_poll; fd++) {
fds[fd].fd = fd + min_fd;
fds[fd].events = POLLIN;
fds[fd].revents = 0;
}
int ready = poll(fds, fd_per_poll, 0);
if (ready == -1) {
perror("poll failed");
return 1;
}
for (int fd = 0; fd < fd_per_poll; fd++) {
if ((fds[fd].revents & POLLNVAL) == 0) {
printf("Fd %d open\n", fd + min_fd);
}
}
}
}
runs through 2^28 file descriptors in 1.6 to 1.8 seconds, which would be just under 7us to check the first 1024 under the same assumptions (though the linear assumption might not hold down to only one loop pass because e.g. warming up the branch predictor and tlb).
It also doesn't depend that much on the RLIMIT_NOFILE value (the limit on the size of a poll call), artificially limiting it to only 256 entries on my machine (such that it makes 4x the syscalls) is only a ~20% performance regression.
It is fully sound if you miss FDs. Those will just never be available via the global FD manager.
iirc all open FDs are guaranteed to be less than the value you get from getrlimit(RLIMIT_NOFILE, ...), so that's not the limit on how many FDs you can pass to one poll call, but the limit on the maximum number you need to check.
I was looking at the man page for poll which says that RLIMIT_NOFILE is the maximum array size. I missed that this is also the limit for the larger fd number, so you're right, this actually only needs to run a single poll call.
I would like to mention that there are situations where you need to close one of these externally sourced fds; the situation that comes up most frequently in real life is that if your program sends data to stdout, it needs to end by closing stdout and checking for errors, lest write errors be silently lost.
(stdout is an extra-special case because of the elaborate song and dance required, in a multithreaded program, to close the open file description and thus receive the errors, without ever letting another thread see an invalid fd 1. But it's the kind of thing that can and should get packed into std::io::Stdout. And if you know there are no other live threads and you're about to exit, most of the rigmarole is unnecessary.)
yes, though just because you can do it all in one poll call doesn't mean you should, e.g. what if the limit is i32::MAX?
iirc if you want to check for write errors, the correct way is generally to fsync it (after flushing any userspace buffers), not close it. Rust intentionally ignores errors when closing files: owned.rs - source
Unfortunately, neither POSIX nor any specific Unix kernel (that I know about) offers any guarantee that a close after a successful fsync (with no other operations in between) will succeed. All the cases I know about, where it could potentially fail, involve network and/or FUSE file servers that are at least arguably buggy, but application authors don't get to demand that the file server gets patched (and there might not even be a patch).
See the long discussion starting here: https://sourceware.org/pipermail/libc-alpha/2026-January/174388.html and spilling briefly into the next month https://sourceware.org/pipermail/libc-alpha/2026-February/174877.html.
EDIT: Also, detecting write errors is not the only case where you need to close an fd that was open on process startup. I don't have any examples to hand, but I have encountered software that would start a subprocess with a communication channel open (on a separate fd from the stdio streams, iirc), expecting the subprocess to close that channel when it was done with it and keep running for some time afterward.
1 Like
Then any program using the manager would have a weird limitation where it can't accept FDs if they happen to be high numbers. That adds significant complexity for whoever is invoking the process.
Yes, by "patched" I meant sending changes upstream, not downstream patching.
I agree that the things I mentioned would ideally be part of libc. But I remember the last time the idea of changing libc was tried. Around 2021-2022, people were looking into the idea of adding new thread-safe environment variable access APIs. Long story short:
- The musl maintainer rejected the idea, blaming Rust for wanting to use setenv from threads (which is sort of reasonable but ignores the massive amount of existing code that does this, not just in Rust but in C, Python, Ruby, etc.). Maybe he could have been convinced, but similar opposition would probably apply to most of the "safer API" type ideas which are fundamentally about protecting buggy code. musl really likes its minimalism.
- For glibc, there were some plans to submit a proposal to them but then it never happened. Not their fault, but who knows what the reception would have been.
- For the proprietary libc owners, Apple, Microsoft, and Google, nobody even contacted them. At least in Apple's case, probably nobody can contact them.
- Even if every libc adopted a new API tomorrow, people would still want to target older OS versions for, in Windows' case, a decade plus. So Rust could not depend on it.
I think prospects are dim for the Rust community ever being able to shepherd a change through that gauntlet.
In contrast, creating a new library, and convincing discrete projects like libwayland (FD passing) or CPython (environment) to adopt it, is at least barely possible. The trick would be finding a way to statically link the library into each project but coordinate between multiple copies of the library linked into the same process, so that nobody has to add a new dynamic library dependency (at least on non-open-source platforms). Which is not pretty, but again, at least it's possible.
To be fair, libwayland itself is only used on Linux and FreeBSD, so fixing libc is a more viable idea for that specifically, but probably there are other libraries that follow a similar pattern and are cross-platform.
1 Like
For the record, I'm still open to the possibility of being hired to make that API happen (and get it standardized, so musl etc. would have to fall in line). I'm not doing it on a volunteer basis, though.
1 Like
Yeah, I agree closing FDs to inform some other process that the other end of the pipe/socket/etc. no longer exists is a valid use case.
I just stumbled upon this, but it seems that you actually posted an example of doing this with stdout in 2017:
In theory, though, you could (1) dup the special fd to a freshly allocated fd, (2) dup2 some other file (e.g. /dev/null) to the special fd, and (3) close the freshly allocated fd. That would let you get the error code from close without ever putting the special fd in an unallocated state (not even transiently).
For the stdio fds, most existing code that closes them is probably replacing them with /dev/null somehow or other (albeit often with a race window). This is because, as you note in the issue, lots of code blindly assumes that those fds are stdio and so it's bad to let them be reused.
For non-stdio special fds, though, I expect most existing code would just close the fd, not seeing it as a problem if the fd is reused.
It turns out that any potential write errors are returned by the next close, rather than the close that fully closes the file when there's no longer any duplicates.
so, a sequence like moved_stdout = dup(stdout), dup2(dev_null, stdout), close(moved_stdout) would have the errors eaten by the implicit close in dup2, and then the close(moved_stdout) wouldn't have any errors left to report.
all that makes me think that trying to close things to get any potential errors is mostly useless because random code will duplicate and then close things (e.g. if you start a new process, using the standard unix fork/exec, the fork will duplicate all FDs, and then the execve will then close all the FDs that the new process isn't supposed to have (marked O_CLOEXEC), obviously with no way to report any write errors)
Yeah on platforms where we can;t scan the whole FD range, FDs bigger than that wouldn't work by default. (We could have an unsafe method to scan the rest of the range if applications want to opt-in to the higher startup cost.)
That's the best we can do at the moment and it is IMO better than the status quo.
Maybe someone can convince the kernel folks to give us a "next used FD bigger than x" or "highest used FD" sycall to make scanning more efficient... or we could use /proc when available so at least most of the time it'd just work.
The global FD manager API would allow that for FDs that you own exclusively.
But closing the standard FDs is clearly unsound as long as any other code may still be running in the current process.
Maybe that's an API we should have for the standard FDs -- a kind of replace.
Indeed, that's why Rust has no API to get the errors returned by close. This was discussed ad nauseam, let's not reopen that discussion in this thread -- this is the wrong place to relitigate that decision.
A kind of replace you say?
3 Likes
Nice 
(A little off topic but) Is there any chance you have a link to the discussion on env::set_var being made unsafe? If I had to guess the only issue with making it safe would be that multithreaded access would cause race conditions; however any environment that offers threading should also offer synchronization primitives that std could take advantage of to serialize accesses and make that a non-issue...
1 Like
Yeah, I don't like the kernel semantics either, and it seems intractable to get them fixed.
Still, if you have at least three spare FD slots open, and you have reason to believe that no other process is currently writing to the open file description backing stdout, which is the normal case for shell utilities and such: I think this sequence will reliably capture all errors that ought to have been reported on one of your previous writes... unless the kernel semantics are "some errors might be delayed until the very last close, across all processes that have this file open", in which case it's just hopeless.
void finish_stdout(void) {
int d1 = -1;
int d2 = -1;
int dn = -1;
bool locked_stdout = false;
#define FAIL(msg) do { perror(msg); goto cleanup; } while (0)
d1 = dup(STDOUT_FILENO);
if (d1 < 0) FAIL("dup");
d2 = dup(STDOUT_FILENO);
if (d2 < 0) FAIL("dup");
dn = open("/dev/null", O_WRONLY);
if (dn < 0) FAIL("/dev/null");
flockfile(stdout); locked_stdout = true;
// this _ought_ to be enough by itself, but ...
if (ferror(stdout) || fflush(stdout)
|| fdatasync(STDOUT_FILENO))
FAIL("stdout: write error");
// catch errors reported on _any_ close, but not before
if (close(d2)) {
d2 = -1; // even if close reports an error, the fd is closed
FAIL("stdout: write error");
}
// atomically replace fd 1 with /dev/null
// it's conceivable that this would trigger error reports for
// data written to fd 1 specifically, and it shouldn't be
// possible for it to fail for any other reason
if (dup2(dn, STDOUT_FILENO))
FAIL("stdout: write error");
(void) close(dn); dn = -1;
// catch errors reported only on the last close from this process
if (close(d1)) {
d1 = -1; // even if close reports an error, the fd is closed
FAIL("stdout: write error");
}
funlockfile(stdout); locked_stdout = false;
return;
cleanup:
if (d1 != -1) close(d1);
if (d2 != -1) close(d2);
if (dn != -1) close(dn);
if (locked_stdout) funlockfile(stdout);
#undef FAIL
}
(rust version left as an exercise 
)