When you are set as a reviewer on a PR by highfive, highfive will ping you on irc in the #rust-bots channel. This only works if highfive does the assigning (i.e., manually assigning someone will not ping them). To get highfive to set a reviewer use r? @nick29581 (either in the commit message or a comment on your own PR), replacing nick29581 with your reviewer of choice. This will only work with reviewers who can be assigned in GitHub.
If you’re a reviewer and want this to work, then you need to have an entry on rustaceans.org, so that highfive can map your GitHub username to your irc nick.
Thanks to sinistersnare for the implementation!