GCC Compile Farm for rustc

community
1

I’ve noticed that some people like me have started to view the GCC Compile Farm as an attractive option for building rustc in the case that you a) don’t have a powerful computer to build on, b) need to build on other architectures, c) other valid reasons. However, it is not terribly straightforward to setup and use the GCC Farm, and the nature of a compile farm presents challenges to the ordinary edit-build-test workflow that most people are used to running locally.

So, I thought I’d share my set-up along with a couple of Bash scripts I wrote to assist with using GCC Farm for rustc.

Firstly, ensure you have registered on the GCC Compile Farm website, been approved, uploaded your public key for SSH, and waited for it to sync to the farm machines. You probably want to edit your SSH configuration to something like:

~/.ssh/gcc-compile-farm

~/.ssh/config (append)

Host gcc*.fsffrance.org
	ControlMaster auto
	ControlPath ~/.ssh/control-%h:%p:%r
	HostName %h
	User <username>
	IdentityFile <keyfile>

Include gcc-compile-farm

Next, when you actually want to use a server, I recommend keeping an active connection to one so that a master control connection is always open an available. This makes the workflow a bit smoother and faster.

Finally, the workflow scripts… The idea is to make the workflow for the developer as analogous as possible to the typical machine-local one. Put these scripts in your path (and make sure they’re executable, of course).

Workflow Scripts

Here’s an example session (assuming in you’re in a rustc repo directory):

$ export GCC_FARM_SERVER=gcc10 # gcc10.fsffrance.org
$ gcc-farm-rustc -s ./x.py build # syncs local dir to the server using rsync, then runs x.py to build
$ gcc-farm-rustc -t ./x.py test # runs x.py to perform all tests on server, then syncs back "blessed" test results using rsync
$ gcc-farm-rustc -s # just syncs local dir
$ gcc-farm-rustc -t # just syncs back test outputs to local dir

The working tree (i.e., current state of the directory) of rustc is in fact synchronised to the server, albeit ignoring all the same files that .gitignore does. Thus, there is no need to commit before syncing.

Note that the gcc-farm script can be used generically, or built upon by another script like how gcc-farm-rustc does. Inspect the scripts for environment variables that can be used to control behaviour.

In any case, your particular needs may deviate somewhat from my above set-up, but I thought I’d post this here as a guide to those getting started, in case it helps anyone. Feel free to post your own suggestions on this thread too.

(Thanks to @dlrobertson for discussing this with me and helping to test my scripts.)

12 Likes
2

@alexreg How long does a build take? Would it be possible to use something like this to speed up CI builds?

1 Like
3

@mark-i-m Depends on the machine you’re using, but anywhere from on par with a good laptop to significantly faster. You can see the specs of the machines on the “list of machines” page of the website. Note that the maintainers instruct you to use only half the number of machine cores at most at one time.

1 Like
4

I wonder if fuzzers are within the intended use of GCC farm. I’ve thought about setting up a rustc fuzzer on a GCC farm box.

1 Like
5

I’ve updated the Gist a bit lately, in case anyone is using them…

1 Like
6

I’m using it and works for me, thank you. The GCC compile farm is great for those that don’t have a very powerful machine.

7

Glad to hear. Yes, it is. Or even for those of us with a decent machine who want to use it for something else relatively CPU-intensive while building rustc.

1 Like
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.