Providing the compiler with semantic reasoning in code path

language design
1

Proposal: #[likely] / #[unlikely] Attributes for Branch and Layout Optimization

Summary

Rust currently provides a few coarse mechanisms to help the compiler reason about branch frequency — #[cold] on functions, core::intrinsics::{likely, unlikely} (nightly), and profile-guided optimization (PGO). These are useful, but they don’t scale elegantly to common patterns such as multi-arm match statements or nested if chains, where the programmer knows which paths are overwhelmingly dominant (“happy path”) and which are rare (“sad path”).

I’d like to explore adding lightweight, first-class attributes such as:

if #[likely] cond { ... } else #[unlikely] { ... }

match result {
    Ok(v)      #[likely(0)]    => process(v),
    Retry      #[likely(1)]    => retry(),
    Timeout    #[likely(0.05)] => backoff(),
    Err(e)     #[unlikely]     => handle(e),
}

The goal is to let programmers express expected execution likelihood in a way the compiler can translate into existing LLVM/Cranelift metadata, improving branch prediction, code layout, cache locality, and pipeline efficiency, without affecting semantics.

Motivation

Programmers often have domain knowledge that static analysis and profiling can’t capture easily — for example:

  • Error or failure paths that occur <1% of the time.
  • Status enums where one or two arms dominate under normal operation.
  • Control loops where the “continue” case is typical and the “break” case is exceptional.

Today we can use intrinsics or code re-ordering tricks, but these are less expressive and inconsistent across if/match constructs.

Explicit hints would help:

  • Shape code layout (hot fall-through first, cold blocks out-of-line).
  • Drive branch-weight metadata (llvm.expect, !prof branch_weights).
  • Influence backend layout (ARM branch hint bits, x86 fall-through, etc.).
  • Improve I-cache and pipeline utilization by keeping hot paths contiguous.

Semantics (sketch)

  • Purely advisory; no effect on program logic.
  • Accepted forms:
  • #[likely] → hot path.
  • #[unlikely] → cold path.
  • #[likely(p)] → numeric hint (p as ordinal or probability).
  • Unannotated arms/branches are neutral.
  • Compiler lowers hints to MIR metadata → backend branch weights → target-specific layout and hints.
  • PGO or JIT data override static hints when available.

Implementation considerations

  • On LLVM backends, this maps cleanly to llvm.expect or !prof branch_weights.
  • On Cranelift, could attach similar metadata to control-flow edges.
  • On x86, this affects block ordering and alignment; on ARM, also sets architectural “taken/not-taken” bits.
  • Can interact naturally with existing #[cold] and inlining heuristics.
  • #[likely]/#[unlikely] on match arms could influence both ordering and outlining decisions.

Example

#[cold]
fn handle_error(e: Error) { log::error!("{:?}", e); }

fn process(x: i32) -> Result<i32, Error> {
    if #[likely(x >= 0)] {
        Ok(x + 1) // hot fall-through
    } else #[unlikely] {
        handle_error(Error::BadArg);
        Err(Error::BadArg)
    }
}

Even on x86 (no explicit hint bits), this would encourage contiguous layout of the hot path and push the error handler into a cold section. On ARM and others, actual branch-hint encodings would be emitted.

Open questions

  1. Should numeric weights (#[likely(0.9)]) be supported, or only binary hints?
  2. How would this interact with #[cold] and inlining thresholds?
  3. Should unannotated arms remain neutral, or implicitly less likely?
  4. What’s the best MIR representation (block metadata vs. edge attributes)?
  5. Is there a feasible path to make this work uniformly across LLVM, Cranelift, and GCC backends?

Why discuss now

LLVM and Cranelift already support branch-weight metadata. Rust developers frequently reach for core::intrinsics::likely/unlikely, but a first-class attribute syntax could make this idiomatic, stable, and portable — especially valuable for embedded and high-performance workloads where layout matters as much as prediction.

Would the language or compiler teams be open to exploring this direction, perhaps starting as an experimental -Z likely-attributes feature?

Links / background

  • #[cold] attribute docs
  • core::intrinsics::likely, unlikely
  • LLVM llvm.expect intrinsic
  • LLVM PGO and BOLT documentation

Closing

This proposal isn’t about micromanaging the predictor; it’s about giving the compiler better priors so it can produce straighter, more cache-friendly instruction streams. Feedback from the compiler and language teams on feasibility, naming, and potential pitfalls would be greatly appreciated.

Note: this markdown was generate by ChatGPT based on our discussions.

2
3

Don't see any mention of core::hint::{likely, unlikely}. Obviously only applicable to if conditions (not match arms etc) but also not intrinsics.

Also cold_path

4

On match, I'd prefer something that doesn't require annotating every arm.

In practice there are a few strategies to implement match: a lookup table (if you're lucky), a linear scan that favors early items the existing order, or bisection that assumes equal probability of all items. And they could be mixed for subsets of the range.

So match just needs a choice of the strategy or an approximate probability distribution function.

1 Like
5

This idea does not require annotating every arm, only as much you feel useful. If you don't annotate, it becomes neutral. In practice, it might make sense to only annotate one arm, depending on the context.

Can you suggest other strategies with match for indicating likelihood?

6

What does neutral mean?

I think in the current implementation, the compiler emits equivalent of linear if/elseif/else for all arms, which allows you to assume that the first match arm will be as fast as possible (implying power law distribution of likelihoods). It may optimize to a table anyway, but if not, it will still check the first arm first, so you can order arms by most likely first. That's O(1) to O(n).

The problem is that the compiler could try to use something fancier instead, like bisection (check if the value is in the first or second half of the arms, recursively, and it may require sorting the arms). It might be faster overall O(log n), but the first couple of arms won't be guaranteed to be the fastest anymore.

I presume that in practice, probabilities of match arms matching will be somewhere between a power law (where there's a most common value dominating) and an even distribution (random, unpredictable).

So I'm not sure how I would annotate all arms as equally likely, given that the current implementation doesn't treat them as equally likely. If I annotate one arm as likely, does it make all other arms as equally less likely?

1 Like
7

Neutral means unweighted.

Yes, the implication is that if you annotate one arm as likely, it makes all other arms equally less likely. That would be my natural intuition.

I want to stress these are not directives to the compiler; they are 'hints'.

We are saying, "as a programmer, this is what I believe is the most likely path(s) the code will take, but you, the compiler, may know more than I do. Hopefully, the CPU Branch Prediction Logic will not undermine both of us."

If Rust ever implements this, it would be interesting to perform a performance test to see if it makes a difference.

8

TBH, I don't know how much it would actually matter, because of all the other things that do exist that mostly cover it:

  • #[cold] so that any path leading to a panic (for example) is automatically unlikely without needing any annotation on the branch
  • select_unpredictable in std::hint - Rust for the places where actually it's neither likely nor unlikely, and you want to avoid branchiness
  • PGO for having better information than the programmer's intuition anyway about which way the branches usually go

TBH, most of the likely/unlikely I see are cargo-culted in ways that they're not obvious at all whether they helped -- and often they hurt things like inlining and other compiler optimizations so can just make things worse.

1 Like