Static binary support in rust

[comex]

How hard would it be to make linking against MUSL or glibc a runtime option? Unlike C code, Rust code shouldn’t have issues with one libc’s headers creating references to symbols not defined in another, so the same standard library .a/.rlibs should be usable with both. (An earlier post states that liballoc somehow references __rawmemchr… anyone know how that gets in there?)

[aidanhs]

I’m strongly against putting musl as the default for the interoperability reasons @alexcrichton outlines - recompiling python (after installing the musl-tools package) to be able to use rust defaults is not a friendly way to interact with C. And it’s not like this is a two-step process anyway - trying it now, all python C extensions have failed to compile with musl-gcc. For a language eager to provide an alternative to C++, I think the defaults should make common C++ tasks (linking to existing system libraries) as easy as possible.

That said, I would like musl to be available by default.

How hard would it be to make linking against MUSL or glibc a runtime option? Unlike C code, Rust code shouldn’t have issues with one libc’s headers creating references to symbols not defined in another, so the same standard library .a/.rlibs should be usable with both. (An earlier post states that liballoc somehow references __rawmemchr… anyone know how that gets in there?)

By runtime, are you talking about ‘compiler’ or ‘application’?

The reference is in r-jemalloc-ctl.pic.o. Looking at rust/src/jemalloc/src/ctl.c it has a reference to strchr, which is defined as a macro on Ubuntu 14.04 in /usr/include/x86_64-linux-gnu/bits/string.h which invokes __rawmemchr. Of course, if this macro was disabled then I guess you’d be risking a slowdown if you do use glibc?

[alexcrichton]

How hard would it be to make linking against MUSL or glibc a runtime option?

I think that this kind of decision would have to be made at compile time, and I would love to make this a super easy option! This would be selected through one of cargo build --target x86_64-unknown-linux-{gnu,musl}, and the only requirement is that the compiler has the target libraries lying around. I would love to have Cargo (or some other tool) provide easy downloads of these target libraries.

Put another way, selecting between glibc/MUSL in my opinion should definitely be as easy as just passing the right --target value. Actually getting the build to succeed is a whole nother story

[comex]

I meant runtime of the compiler (misread a comment as saying it would be a rustc configure option). But specifically, it would be nice to be able to build with musl without having to download and use a separate set of libraries, aside from musl itself, which should be small enough to just include in the normal distribution.

[alexcrichton]

it would be nice to be able to build with musl without having to download and use a separate set of libraries, aside from musl itself, which should be small enough to just include in the normal distribution.

In theory this is possible because we don’t link against the glibc headers, but rather via symbols, but it would require some significant compiler internal specialization and also overhaul how the MUSL target actually works. It also wouldn’t enable native dependencies to understand when they’re linking against MUSL or not, which may not work out so well unfortunate.

[gus]

A lot of complex issues are being oversimplified-to-the-point-of-incorrect in this thread - I’d caution against taking any actions without further research.

For example, go-lang does not statically link libc - it does something very similar to what rustc currently produces:

% ldd $random_go_executable linux-vdso.so.1 (0x00007ffd14dc0000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f289bfa0000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f289bbf7000) /lib64/ld-linux-x86-64.so.2 (0x00007f289c1db000)

[burntsushi]

For example, go-lang does not statically link libc

It does on my machine.

[andrew@Serval cmail] go version
go version go1.4.2 linux/amd64
[andrew@Serval cmail] ls
forever-out  main.go  README.md  UNLICENSE
[andrew@Serval cmail] go build
[andrew@Serval cmail] ls
cmail  forever-out  main.go  README.md  UNLICENSE
[andrew@Serval cmail] ldd cmail
        not a dynamic executable

[steveklabnik]

Are you sure you’re not talking about gccgo? The official Go compiler doesn’t use libc at all, I thought.

[aidanhs]

Ignoring gccgo briefly, I believe you only get dynamic linking in go when the compiler’s hand is forced, typically by nsswitch…and even then you can get static linking with sufficient knowledge of go compiler flags (as docker does).

Static linking doesn’t necessarily mean statically linking libc though - “Gc uses a custom C library to keep the footprint under control”, which presumably means it doesn’t actually link libc at all (except in cases where it needs to, per above).

[gus]

No, not gccgo - the regular go compiler links dynamically if you call anything that uses nss (ie: just about any non-trivial program).

Eg, this program links libc dynamically:

package main
import "fmt"
import "net"
import "os"

func main() {
        h, err := net.LookupHost(os.Args[1])
        if err != nil {
           fmt.Println("Failed:", err)
        } else {
           fmt.Println("result:", h[0])
        }
}

And:

/tmp% go build test.go
/tmp% ldd test        
    linux-vdso.so.1 (0x00007ffca55ed000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7e44e29000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7e44a80000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f7e45064000)

My point is only that the details are more complicated than has been summarised in the above thread.

[aidanhs]

Just as you can link nss statically with a C program (even with glibc), so can with go. Per my previous post, docker does it and it seems well supported.

I’m not sure what details have been simplified though. It’s a non-trivial problem, which is exactly why I created this thread.

This evening I’ll try and put my money where my mouth is and write up some of my notes on the current state of static linking glibc in rust.

[cmr]

but iOS does not support shared libraries at all.

This is false, as of iOS 8.

---

There’s a lot of nuance to library linkage, generalizations are pretty useless. It’s per-platform, per-version, and can change based on your dependencies and usecases. We should keep our avenues open and try and do what everyone else does, on each platform.

[jimmycuadra]

This topic (and Rust in general) are way more low level than things I’ve experienced previously, so my understanding is very low, but what is the fundamental difference between Rust and Go that prevents Rust from creating binaries that don’t dynamically link to glibc?

In Linking golang statically, the author statically links glibc with a “glibc-static” package, which, according to everything I’ve read on this topic in the Rust world, says is either not possible or extremely ill-advised. It’s probably my lack of general knowledge on this type of systems programming, but I’m not clear about why Rust can’t take this same approach. The Rust team has been working on support for using musl instead of glibc, but Go has not had to do this to achieve static binaries, AFAIK.

My use case is creating Docker images that contain only static binaries with no operating system, like this:

FROM scratch
COPY ./rust_program /rust_program
CMD ["/rust_program"]

…where rust_program is just a binary that was compiled on any system with a Linux kernel. This is a pattern that is used and hyped a lot by people like Kelsey Hightower and I really want to be able to write Rust programs I can distribute this way. In most of Kelsey’s examples he just uses CGO_ENABLED=0 when building and that’s all it takes.

[steveklabnik]

what is the fundamental difference between Rust and Go that prevents Rust from creating binaries that don’t dynamically link to glibc?

Go does not use libc, they re-implemented all the funcationality themselves.

[aidanhs]

Linking to glibc-static is neither impossible nor ill-advised nor unsupported, it just requires you to be aware of the subtleties of NSS. If you do not need NSS or you enable static NSS when compiling glibc, statically linking glibc is fine.

In the blog post you refer to, they link against a C program which does not need NSS so all is well. If they were to try and link statically against Python it could fail in mysterious ways inside a container.

Go gets away with this in general because linking to existing C libraries is not popular - reimplementing in pure go is much more common. And so because, as @steveklabnik says, the go runtime don’t use libc at all there’s no need to link against it. One obvious tradeoff is that go therefore doesn’t use NSS. Try and make the Docker socket be owned by an LDAP user - it won’t work, because go won’t use nsswitch.conf. I had to write a patch to allow specifying ids to work around this.

Coming back to Rust, I (and others) use musl and it works pretty well. Just follow the instructions in the nightly docs.

[jimmycuadra]

Thanks for the responses, they were very helpful!

I hadn’t discovered the advanced linking section in the book because I was looking at the stable docs which only talk about the link_args feature.

I opened an issue on rust-lang/rust about an official binary distribution of Rust with musl enabled, since the Rust 1.1.0 blog post suggested they were going to make one. Does anyone know if this is still planned or why it hasn’t happened yet? As far as I can tell, the only reason this documentation is included in the “Nightly Rust” section of the book is because you have to build the compiler yourself, not because it depends on any unstable features. I’d like to see a musl-enabled version of Rust on https://www.rust-lang.org/install.html as an alternative option for Linux because of its usefulness and to make it clear that Rust can do this! It’s a big language selling point, IMO.

[jimmycuadra]

Go does not use libc, they re-implemented all the funcationality themselves.

I found more information on why Rust didn’t do this (and probably won’t in the foreseeable future) from Erick in a Hacker News comment responding to the question of why Rust needs libc:

No it’s not a stupid question. libc (or CRT on windows) really is the library that exposes all the user space system libraries. It contains the functions to do IO, sockets, threads, and etc. So we use it to expose that functionality to rust users.

Now there are some languages, namely Go, that skip libc and just implement directly against the syscall interface. Go has the advantage of being able to draw from Google’s vast experience interacting deep within the system, so it was comparatively cheap for them to do this.

For rust, it never really felt like it was worth the effort for the benefit we’d get out of it. It was more important to get the language done.

I saw a comment from Steve on HN saying effectively the same thing when I was Googling around earlier, but now I can’t find it.

[aidanhs]

IIRC the liblibc crate within rust was broken for musl in Rust 1.1, so musl builds for that version may be broken in surprising ways.

Builds seem to be planned, but I don’t know what the current priorities are. Though I’ve just discovered the buildbot configuration, so potentially all I need to do is make a PR there…

[jimmycuadra]

There already seems to be a musl build but I’m not sure if this is directly related to what gets added to https://static.rust-lang.org/dist/, and certainly it’s not related to the main download options shown on https://www.rust-lang.org/install.html.

[steveklabnik]

I saw a comment from Steve on HN saying effectively the same thing when I was Googling around earlier, but now I can’t find it.

It’s also just things like bugfixes, too. Just like any other project, you can write it yourself, or you can use a dependency. Pros and cons to both.

I will also say (this might have been mentioned above, sorry if I’m repeating) that in Go, lots of people build everything in go, which means interop with C stuff is much lower priority. Us having good interop with C and C libraries is very important, and lots of them are built on libc. This affects other things too, like threading models.