Upcoming Automation Changes

internals
1

Over the past few weeks @barosl has performed an awesome rewrite of bors called homu and it’s had some pretty good traction for the cargo repository. I plan on now unleashing homu onto the rust-lang/rust repository itself soon, but I wanted to give everyone a heads up as there are indeed some “breaking changes”.

Why rewrite bors?

The architecture of the old bors script was to run periodically as a cron job. Each time it ran bors would poll the state of the world and then move the world forward one step at a time. While certainly a simple idea (and it sure took us far!) this had some downsides:

  • Bors gets slower as the queue gets larger. Because bors has to poll ever PR, the runtime is linear with respect to the number of open PRs, which means that bors is slowest when we need him most and we’re often hitting the github API rate limits.
  • Transitions between PRs take quite a long time. Due to the rate limiting in the bullet above bors was run only once every 10 minutes. To transition PRs bors had to (1) discover tests passed, (2) merge the PR, and finally (3) find the next PR to merge. This meant that transitions between PRs could take up to 30 minutes!

Altogether, we found that the architecture of bors was too limiting, and enter homu!

How is homu different?

The primary change between bors and homu is that homu is event driven through the github webook API and the buildbot HTTP push API. Homu is not a cron job but rather a long-running server which isn’t intended to be shut down that frequently. It works by receiving status updates from github and buildbot as soon as they happen allowing for homu to make transitions near-instantaneously!

By using these push APIs from github and buildbot homu is able to not slow down as the queue gets larger and continue to move everything along efficiently.

What’s changing?

The old bors status page will no longer be updated, and the new homu status page will be the “one true interface” to the PR queue of Rust. Note that currently homu is not enabled for Rust, only for Cargo (homu supports multiple repos!).

Additionally, the method of approving a PR is changing. The old method of “r+ on the commit in $user/rust” unfortunately doesn’t work for homu as there’s no way to receive push notifications about commits in external repositories. Instead, homu requires the comments @bors: r+ $SHA to be written on the PR itself. All “commands” to bors need to be prefixed with @bors, and other commands include retry, rollup, r-, etc. When the transition is made I will transfer all pending r+ comments from commit comments to the PR itself to ensure that nothing is lost.

In summary:

Timeframe

Currently homu is only enabled for the Cargo repository, not for Rust. I plan on enabling homu for Rust over the weekend (and shutting off the bors cron job), and I will make a post to this thread when I have done so.

Questions? Issues?

Feel free to ask any questions of me, @barosl, or @brson in this thread, and also feel free to open issues against homu itself at https://github.com/barosl/homu/issues!

2 Likes
2

Is the full SHA necessary, or can conventional "short SHA" prefixes be used as well?

3

Short SHAs are fine, homu will also tell you if it didn’t recognize $SHA and even recommend the head SHA to r+. I’ve been copy/pasting the SHA that github shows on the PR page basically and it works just fine.

4

Cool! Sounds like homu is going to be really awesome :sparkles:

5

Fantastic!

How does r- work now? Do you specify the SHA or does it prevent any r+'s from ever working until removed?

6

Looks like it’s just @bors: r- which will cancel out any previous @bors: r+ $SHA.

7

Why keep the name bors in @bors: foo?

8

Just because the software is different doesn’t mean it’s not still bors :slight_smile:

1 Like
9

As to why a prefix is needed at all, Homu primarily reads PR comments where it’s easy to colloquially refer to an “r+” which could accidentally be interpreted as an actual approval. The prefix of @bors ensures that you’re actually giving a command to bors.

As to “bors”, I think @steveklabnik put it well :slight_smile:

10

So, homu is just a code name for the re-write then? For example, bors v2 is aka homu?

11

Actually, I found an incident where an approval was “accidentally given” on a clearly stupid (but funny) commit in a pull request on September 4th.

The hilarious code is currently still in the code base

#[doc = "> The first principle is that you must not fool yourself — and"]
#[doc = "> you are the easiest person to fool."]
#[doc = "> "]
#[doc = "> - Richard Feynman"]
flags Flags: u32 {
    const FlagA       = 0b00000001,
    #[doc = "<pcwalton> macros are way better at generating code than trans is"]
    const FlagB       = 0b00000010,
    const FlagC       = 0b00000100,
    #[doc = "* cmr bed"]
    #[doc = "* strcat table"]
    #[doc = "<strcat> wait what?"]
    const FlagABC     = FlagA.bits
                       | FlagB.bits
                       | FlagC.bits,
}
12

@theme, that is in a test, and is just testing that attributes are parsed in the bitflags macro (as the commit message says); maybe the content of those attributes is a bit funny, but it is not a stupid commit.

13

That sounds about right!

14

Homu is now live!

http://buildbot.rust-lang.org/homu/queue/rust

4 Likes
15

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.