Let `include!` macro join paths

The problem

Currently the include! macro takes a single argument (a path to file), meaning that if you want to join paths you have to do it manually. E.g.:

include!(concat!(env!("OUT_DIR"), "/repro.rs"));

This has problems because you need to assume something about path separators. This is warned about in the docs:

The included file is located relative to the current file (similarly to how modules are found). The provided path is interpreted in a platform-specific way at compile time. So, for instance, an invocation with a Windows path containing backslashes \ would not compile correctly on Unix.

While most Windows paths can use either \ or / as path separators (so / is a bit more cross-platform) this is not true of verbatim paths (i.e. paths starting with \\?\).

Proposed Solution

Allow include! to take multiple arguments, which it will then join in a host dependent manner. For example:

include!(env!("OUT_DIR"), "repro.rs");
// Or with multiple sub-components
include!(env!("OUT_DIR"), "src", "repro.rs");

If given a subdirectory that includes /, it will be treated as a path separator. E.g. this will work cross-platform:

include!(env!("OUT_DIR"), "src/repro.rs");

Alternatives

We could automatically try to "fix" paths that are naively concatenated. This strikes me as a lot more brittle than just doing it properly to begin with.

We could adapt concat! or add a new concat_path! macro to do the path joining.

We could just do nothing. But this is a part of the language that's (imho) unnecessarily platform-dependent. I think we should strive to do better.

5 Likes

This sounds like an excellent idea. I don't know much about Windows systems and as such anything that makes things just work (TM) cross platform would be nice. As far as I know on modern systems it is just the *nix vs Windows ways of doing paths[1].

I assume that if a Rust project is compiled on a network path on Windows it would currently break, as iirc UNC paths don't allow for / as the separator?


  1. Fun side fact, classic pre-OS X MacOS used : as the directory separator, to go up one level you used :: (two separators in a row). Relative paths were those that started with a : separator for the current directory, and absolute paths were those that started with a volume name (e.g. Macintosh HD:System Folder:Finder). At least we don't have to deal with such nonsense any more (though the drive letters are almost as weird)! ↩︎

1 Like

A network path (otherwise known as a UNC path) is something like \\serverName\shareName\path. these can use either forward or backward slashes.

What Rust calls a "verbatim path" (paths starting with \\?\) can only use \ as path separators. These kinda of paths are most encountered when they're because they returned by std::fs::canonicalize. Confusingly some people call these paths "UNC" too, probably because \\server\share is canonicalized as \\?\UNC\server\share.

I describe this in a bit more detail here: Understanding Windows Paths - help - The Rust Programming Language Forum

2 Likes

I think adding a concat_path! macro makes the most sense here, to abstract over the path separator. That would avoid embedding concatenation logic in include and making it unavailable to other use cases.

2 Likes

The only thing that makes me reticent about that is the host vs. target path thing. By bundling it in include it would only be doing host stuff whereas concat_path! would be pulling double duty for host and target paths.

I guess this could be alright if it uses simplified rules for joining.

I would prefer something bespoke to include! / include_bytes! / include_str! over a separate concat_path!. The include macros inherently pertain only to host paths, whereas a concat_path! would certainly end up being misused for target paths, and trying to do double duty will constrain how correctly it can behave for both cases.

concat_path! for target paths is already available on crates·io as the path_macro crate.

"That would avoid embedding concatenation logic in include and making it unavailable to other use cases" — But even with a concat_path! macro its concatenation logic would still continue to be unavailable to other compile-time use cases other than include, because anything else still could not do eager evaluation of the concat_path! expression. For example #[template(path = concat_path!(…))] doesn't "just work". Use cases outside of include (mostly proc macros I guess) need built-in handling of path concatenation which concat_path! does not help with, so it might as well be built in to include too.

7 Likes

Actually, this might be simpler than I first thought as PathBuf already has code to handle this for Windows verbatim paths. I'll experiment a bit.

Ok, I've had a go at making the simpler change to get this working, It just uses components().collect() to fixup verbatim paths: https://github.com/rust-lang/rust/pull/125205