Static binary support in rust

No, not gccgo - the regular go compiler links dynamically if you call anything that uses nss (ie: just about any non-trivial program).

Eg, this program links libc dynamically:

package main
import "fmt"
import "net"
import "os"

func main() {
        h, err := net.LookupHost(os.Args[1])
        if err != nil {
           fmt.Println("Failed:", err)
        } else {
           fmt.Println("result:", h[0])
        }
}

And:

/tmp% go build test.go
/tmp% ldd test        
    linux-vdso.so.1 (0x00007ffca55ed000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7e44e29000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7e44a80000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f7e45064000)

My point is only that the details are more complicated than has been summarised in the above thread.

Just as you can link nss statically with a C program (even with glibc), so can with go. Per my previous post, docker does it and it seems well supported.

I’m not sure what details have been simplified though. It’s a non-trivial problem, which is exactly why I created this thread.

This evening I’ll try and put my money where my mouth is and write up some of my notes on the current state of static linking glibc in rust.

1 Like

This is false, as of iOS 8.


There's a lot of nuance to library linkage, generalizations are pretty useless. It's per-platform, per-version, and can change based on your dependencies and usecases. We should keep our avenues open and try and do what everyone else does, on each platform.

1 Like

This topic (and Rust in general) are way more low level than things I’ve experienced previously, so my understanding is very low, but what is the fundamental difference between Rust and Go that prevents Rust from creating binaries that don’t dynamically link to glibc?

In Linking golang statically, the author statically links glibc with a “glibc-static” package, which, according to everything I’ve read on this topic in the Rust world, says is either not possible or extremely ill-advised. It’s probably my lack of general knowledge on this type of systems programming, but I’m not clear about why Rust can’t take this same approach. The Rust team has been working on support for using musl instead of glibc, but Go has not had to do this to achieve static binaries, AFAIK.

My use case is creating Docker images that contain only static binaries with no operating system, like this:

FROM scratch
COPY ./rust_program /rust_program
CMD ["/rust_program"]

…where rust_program is just a binary that was compiled on any system with a Linux kernel. This is a pattern that is used and hyped a lot by people like Kelsey Hightower and I really want to be able to write Rust programs I can distribute this way. In most of Kelsey’s examples he just uses CGO_ENABLED=0 when building and that’s all it takes.

Go does not use libc, they re-implemented all the funcationality themselves.

Linking to glibc-static is neither impossible nor ill-advised nor unsupported, it just requires you to be aware of the subtleties of NSS. If you do not need NSS or you enable static NSS when compiling glibc, statically linking glibc is fine.

In the blog post you refer to, they link against a C program which does not need NSS so all is well. If they were to try and link statically against Python it could fail in mysterious ways inside a container.

Go gets away with this in general because linking to existing C libraries is not popular - reimplementing in pure go is much more common. And so because, as @steveklabnik says, the go runtime don’t use libc at all there’s no need to link against it. One obvious tradeoff is that go therefore doesn’t use NSS. Try and make the Docker socket be owned by an LDAP user - it won’t work, because go won’t use nsswitch.conf. I had to write a patch to allow specifying ids to work around this.

Coming back to Rust, I (and others) use musl and it works pretty well. Just follow the instructions in the nightly docs.

1 Like

Thanks for the responses, they were very helpful!

I hadn’t discovered the advanced linking section in the book because I was looking at the stable docs which only talk about the link_args feature.

I opened an issue on rust-lang/rust about an official binary distribution of Rust with musl enabled, since the Rust 1.1.0 blog post suggested they were going to make one. Does anyone know if this is still planned or why it hasn’t happened yet? As far as I can tell, the only reason this documentation is included in the “Nightly Rust” section of the book is because you have to build the compiler yourself, not because it depends on any unstable features. I’d like to see a musl-enabled version of Rust on https://www.rust-lang.org/install.html as an alternative option for Linux because of its usefulness and to make it clear that Rust can do this! It’s a big language selling point, IMO.

I found more information on why Rust didn't do this (and probably won't in the foreseeable future) from Erick in a Hacker News comment responding to the question of why Rust needs libc:

No it's not a stupid question. libc (or CRT on windows) really is the library that exposes all the user space system libraries. It contains the functions to do IO, sockets, threads, and etc. So we use it to expose that functionality to rust users.

Now there are some languages, namely Go, that skip libc and just implement directly against the syscall interface. Go has the advantage of being able to draw from Google's vast experience interacting deep within the system, so it was comparatively cheap for them to do this.

For rust, it never really felt like it was worth the effort for the benefit we'd get out of it. It was more important to get the language done.

I saw a comment from Steve on HN saying effectively the same thing when I was Googling around earlier, but now I can't find it.

1 Like

IIRC the liblibc crate within rust was broken for musl in Rust 1.1, so musl builds for that version may be broken in surprising ways.

Builds seem to be planned, but I don’t know what the current priorities are. Though I’ve just discovered the buildbot configuration, so potentially all I need to do is make a PR there…

There already seems to be a musl build but I’m not sure if this is directly related to what gets added to https://static.rust-lang.org/dist/, and certainly it’s not related to the main download options shown on https://www.rust-lang.org/install.html.

I saw a comment from Steve on HN saying effectively the same thing when I was Googling around earlier, but now I can't find it.

It's also just things like bugfixes, too. Just like any other project, you can write it yourself, or you can use a dependency. Pros and cons to both.

I will also say (this might have been mentioned above, sorry if I'm repeating) that in Go, lots of people build everything in go, which means interop with C stuff is much lower priority. Us having good interop with C and C libraries is very important, and lots of them are built on libc. This affects other things too, like threading models.

[quote=“jimmycuadra, post:37, topic:2011”][quote] No it’s not a stupid question. libc (or CRT on windows) really is the library that exposes all the user space system libraries. It contains the functions to do IO, sockets, threads, and etc. So we use it to expose that functionality to rust users. [/quote][/quote] Just clarifying that the CRT on Windows is not the library that exposes the user space system libraries. Windows provides its own set of DLLs that provide all the system stuff and the CRT is just a layer on top to provide the C standard library which Rust only uses for math functions, the entry point, and a few memory functions like memcpy.

Is the desire to put exactly one file into the container really worth the trouble? Isn't producing one self-sufficient file, like, the container's core competency?

It's an approach that effectively uses the container as a packaging format, along with the runtime isolation and resource constraints offered by namespaces and cgroups that you wouldn't get if you were executing the binary directly on the host machine. It's the primary application deployment strategy for projects like CoreOS and Kubernetes. Everything is in a container and all the orchestration tools are built around manipulating containers and scheduling them across a cluster. I use a different Docker image to actually build the program and produce the binary. This means that the runtime container is much smaller because it doesn't contain an OS and all the build tools. The space savings between the two is quite significant when you start to have lots of applications in your cluster.

You’re talking space savings compared to putting the actually used dynamic libraries in there and nothing else, right?

That's actually not something I've seen commonly in the Docker ecosystem today. Most of the time people are just using a slimmed down OS image (commonly Debian) as the base. Technically unnecessary, done either out of convenience for simplifying the build process or pure ignorance. This silliness aside, I still think it's very worthwhile to pursue static binaries in Rust.

I’m bumping this again in the hopes of getting the attention of whomever has control of the build and release process for Rust. I’d really like to see a musl-targeting rustc binary for Linux as one of the install options on https://www.rust-lang.org/install.html. (Discussion moved from https://github.com/rust-lang/rust/issues/27933 per Steve’s request.)

/cc @alexcrichton

You’re waiting for this issue I think - now llvm 3.7 has been released there’s no need to engage in svn checkout madness, musl rust can theoretically be set up as a first-class target on the buildbots. It’s also now even easier to build your own!

Unfortunately the rust buildbots are not fully documented so it’s a little tricky if you were to want to help - I understand it’s on a todo list.

Don't worry, we haven't forgotten about this! We're still working on getting the infrastructure in shape to get to the point where we can easily ship new targets and you can install them super easily as well.

In the meantime, though, you can either build MUSL from source (instructions) or I've also started playing around with Docker a lot recently and I've created a bunch of images which should have things like MUSL or Android installed and ready to go by default: