Improved cross platform std::fs

libs
1

std::fs mostly just works out of the box on all rust supported platforms (with consistent behaviour). There are however still subtle differences especially around symlinks and canonicalization (and maybe others I'm not aware of) that require third party crates to work around (symlink and dunce come to mind). Since these are quite tricky topics that are hard to get right, I think it would make sense to fold those crates into std::fs.

2

For reference:

3

Ok, so to state that in terms of concrete proposals:

Symlinks

symlink_dir and symlink_file could be made cross platform by forwarding to std::os::unix::fs::symlink on *nix platforms.

Canonicalization

I think you want a way to convert from, e.g. \\?\C:\Path\to\file to C:\Path\to\file that takes care of all the intricacies? This is certainly doable but designing the API is made more interesting because the conversion can be lossy.

Note that there is the nightly std::path::absolute if you just want an absolute path without resolving symlinks. This will not return \\?\ paths (unless you give it such a path).

4

well, there is more to it, like removing symlinks. on unix it's just removing a file but on windows it depends on what kind of symlink it is. I noticed that crates like zip-rs don't handle symlinks properly and just creates a regular file. I think that this is likely a symptom of rust std not having a cross platform abstraction, although for this particular case you can select symlink_file/symlink_dir based on path.is_dir().

5

also what happens if you have a symlink to a symlink on windows? it's quite common on linux, not really familiar enough with windows to know. Maybe my frustration is less with rust but with windows being weird.

6

Ah I see, so a better proposal would be something like:

Symlinks

  • A symlink function where the target must exist (so the Windows implementation can figure out which type of symlink to create).
  • A remove_link function that can remove any type of link.
1 Like
7

This would likely be racy.

8

Sure. I'm not certain to what extent that matters in this case. It's only being used a heuristic to guess which type to create. The worst that can happen is the symlink fails to be created and returns an error.

9

yes I think that would help dealing with symlinks. the question is probably if this is sufficiently common to warrant putting it in std, but since std already deals with symlinks the answer is likely yes.

when it comes to canonicalization, std::fs::canonicalize returns a unc path by default which causes issues. would be nice if it returned a regular path if possible. often you want to resolve symlinks from the file system, so std::path::absolute is not a drop in replacement. besides if the path is going to be used multiple times, it's probably what you want anyway.

10

The trouble is some people depend on canonicalize returning verbatim paths. Starting a path with \\?\ has certain special properties like enabling extended length paths or otherwise allowing access to files that would otherwise not be accessible when using drive paths.

Converting the \\?\ prefix may affect these uses so it would need to be opt-in.

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.