I was recently working on large file support in another project, and thought I should see how Rust fares. In fact LFS in Rust appears almost non-existent. A few platforms do use foo64 link names, but in general it appears to not really be considered. So, I’d like to fix this, but want to ping opinions here before I draft an RFC. That is, if an RFC is even necessary – this could be considered just bugfixing.
For the uninitiated, Large File Support (LFS) in C is basically about making sure off_t and ino_t are 64-bit, as well as related compound types like struct stat and related libc functions. This is only a concern for 32-bit platforms; 64-bit code should always be fine. It’s also an issue solved long ago – see here that Linux has had LFS since kernel 2.4 and glibc 2.2.
Some platforms like FreeBSD went ahead and converted to 64-bit off_t etc. all at once. On 32-bit Linux with glibc, the default is still the old 32-bit off_t, and you can opt in with -D_FILE_OFFSET_BITS=64 to get the 64-bit types and functions swapped in-place. I’m not sure what OSX and Windows have. In an autoconf project, you can use AC_SYS_LARGEFILE to set the necessary #defines automatically. It’s a little sad you still have to deal with this at all, but I guess backwards compat with 32-bit off_t programs was considered important enough, and anyway that ship has sailed.
I think Rust should just do the right thing by default, and always use the proper 64-bit types and interfaces. There may be some concerns about changing exposed types post-1.0, like a lot of what MetadataExt returns, but I think there’s strong justification for a breaking change here. The potential for breaking is why I think this might need an RFC. There’s also the libc crate to consider, but again I think the best answer is to expose only 64-bit file interfaces there. (Not to explicitly name fstat64 etc. – just call it fstat but link to fstat64 like what _FILE_OFFSET_BITS would get you.)
Now here’s a concrete example:
fn main() {
for arg in std::env::args().skip(1) {
match std::fs::metadata(&arg) {
Err(e) => println!("Can't stat '{}': {}", arg, e),
Ok(m) => println!("path:'{}' len:{}", arg, m.len()),
}
}
}
Linux x86_64 is fine:
$ truncate -s 1G foo
$ truncate -s 2G bar
$ rustc -Vv
rustc 1.5.0-nightly (9d3e79ad3 2015-10-10)
binary: rustc
commit-hash: 9d3e79ad3728f6723f8e49908696e66a68d7db95
commit-date: 2015-10-10
host: x86_64-unknown-linux-gnu
release: 1.5.0-nightly
$ rustc metadata.rs && ./metadata foo bar
path:'foo' len:1073741824
path:'bar' len:2147483648
But on i686 Linux with the same test files:
$ rustc -Vv
rustc 1.5.0-nightly (9d3e79ad3 2015-10-10)
binary: rustc
commit-hash: 9d3e79ad3728f6723f8e49908696e66a68d7db95
commit-date: 2015-10-10
host: i686-unknown-linux-gnu
release: 1.5.0-nightly
$ rustc metadata.rs && ./metadata foo bar
path:'foo' len:1073741824
Can't stat 'bar': Value too large for defined data type (os error 75)
It still should be sorted out for the libc crate though, right? It looks like 
