Currently in fs::copy, there is no way to set the mode of the file you are creating. This seems like something that would be quite intuitive to have, especially when building lower level utilities, and it also provides, perhaps at first hidden, security benefits.
The lack of an option to set a mode during copying has resulted in inelegant hacks like this (by yours truly) in software that needs to set file permissions before copying data over (to avoid permission races).
What do people think about this? I don't think it would be a very significant change, but I do think it could have quite positive effects on security.
It seems to me that fs::copy is a convenience function for the simplest, most quick-n-dirty use case, and as the documentation notes, you'll probably want io::copy if you need more control over file creation or platform-dependent stuff like permissions.
No, fs::copy can do filesystem cloning or kernel-space-only copying that io::copy can't, it really is its own operation. I'm not sure adding a mode set here is the right choice, but substituting io::copy is a behavior change and potential pessimization.
On Unixy systems, at the system call level, "filesystem cloning or kernel-space-only copying" is still an operation on a pair of file descriptors, not a pair of pathnames. On all supported Unixy systems except MacOS, fs::copy already just calls io::copy, and, skimming the code, I don't see a good reason why the MacOS implementation of fs::copy couldn't be pushed down into io::copy.
On Windows, the CopyFile family of functions really does operate on pathnames, not file handles. I wasn't able to find an equivalent that operates on file handles in a couple minutes skimming MSDN, but that doesn't mean there isn't one. However, CopyFile doesn't appear to allow you to control the ACL of the destination file, either, so the feature you want may not be possible to implement for Windows. (People who know more about low-level Windows programming than I do, please correct me if I'm wrong about anything in this paragraph.)
Therefore, I think the Right Thing would be to push the MacOS-specific fs::copy down into io::copy, make a documented guarantee that io::copy will use copy_file_range or equivalent primitive whenever possible, and also document in fs::copy that if you want to control the ownership, permissions, or other metadata of the destination file, create it yourself and use io::copy.
On Windows you're correct, there's no single function to copy using file handles. You can implement this manually (which is essentially what CopyFile does) but that's more involved. There are some low-level functions which can help, such as NtCopyFileChunk, but that doesn't copy metadata itself and it doesn't work in all cases. Quoting from those docs:
NtCopyFileChunk is used internally by CopyFile for most forms of copy. Current exceptions include cloud copies, SMB offload and ODX.
As mentioned by zackw, io::copy definitely can do those. The implementation literally first tries a function called kernel_copy and falls back to generic copying if that doesn’t work. Specifically on Linux the docs note that
On Linux (including Android), this function uses copy_file_range(2) , sendfile(2) or splice(2) syscalls to move data directly between file descriptors if possible.
Linux's copy_file_range also doesn't do the whole job and doesn't work in all cases; callers need to check for a "can't do it this time" error return and fall back to a manual copy loop. It sounds like the Windows implementation of io::copy might usefully be taught to use NtCopyFileChunk if it doesn't already, but fs::copy on Windows should keep using CopyFile, would you agree?
The Mac fclonefileat operates on a directory descriptor + basename for the destination, not a file descriptor. Which makes sense, since it is modifying directory entries rather than file contents. That said, I'm also not sure it can modify permissions atomically with the clone—the manpage suggests you can choose "same as original, if permitted" or "reset to the default for the destination directory". Which means changing the fs API wouldn't be able to provide atomicity anyway.
EDIT: …and copyfile now has a COPYFILE_CLONE flag, so maybe fclonefileat is a distraction anyway.
Which it should not yet, because if copy_file_range fails, splice is still likely going to work.
The manual pages don't make it exactly clear what the difference is, but given the limitations mentioned and my understanding of NFS:
copy_file_range works at the filesystem level. It only works on pairs of regular files on the same filesystem, but it can take advantage of any copy-on-write or snapshot capability of the filesystem, or in case of NFS 4.2 issue the request for the copy to occur fully server-side (well, if it's already implemented; I didn't check that, I only know NFS 4.2 has that function).
splice (which has the same function signature) works at VFS level. It issues read to the one file and write to the other, just without copying the data to userland and back. That way it works between files, sockets and pipes, and between files across filesystems, but it always copies the data and it always fetches the data and uploads them back in case of a network filesystem.
So the correct approach would really be to try copy_file_range, but then fall back to splice, not a manual copy loop (I don't see a way for splice to fail and manual copy loop still work so that second fallback shouldn't be needed).
splice only works if at least one of the specified file descriptors refers to a pipe – it won't be able to copy between two regular files on unrelated filesystems.
It would be possible to implement a copy by splicing from one file to the other via a pipe (which may require several system calls because a pipe has a limited capacity). I don't know how this compares performance-wise with a manual copy (although I'd expect it to be a bit faster).
Another option is sendfile, which (in recent kernels) has the restriction that the file being copied from is mmap-able (or that the call can be desugared into splice). This would be able to handle almost all combinations of files, but not quite all of them (e.g. if both the input and output are sockets, none of sendfile, splice or copy_file_range would work).
Yeah, you are right, I overlooked that, and it seems to still apply (as the other functions had their constraints relaxed over time).
It would be possible to run two splices in separate threads in parallel, or let kernel threads run them via io_uring, but that would be quite a lot of work for a relatively rare operation.
That does seem like a nice potential simplification. COPYFILE_CLONE was added in copyfile-138 which is from macOS 10.12, and 10.12 happens to be the minimum macOS supported by Rust. So the Apple fs::copy implementation could probably be replaced in its entirety with a call to copyfile.
But just to be clear, copyfile and fcopyfile are userland library functions, not syscalls, so they don't add any new abilities. In particular, fcopyfile doesn't let you clone into a file descriptor. The implementation of COPYFILE_CLONE is based on clonefileat and only works with regular copyfile. fcopyfile seems to just silently ignore the flag (even if you pass the _FORCE variant!).
macOS doesn't expose syscalls as part of its stable interface. You may be right about their implementation today not providing the same behavior, but it's not generally true on macOS that a "userland library function" doesn't turn around to perform a syscall.