Pre-RFC: Stabilization of `likely(bool)`, `unlikely(bool)` & `assume(bool)` intrinsics

[the8472]

@matthieum

First of all, the likely and unlikely hints do not, in general, have any effect on the branch predictor. They are purely about code-placement: the code of the likely branch will followed the check immediately, while the code of the unlikely branch will require a jmp, and may thus suffer an instruction cache miss.

My understanding is that in practice this placement leads results in an effect on branch predictors because - absent a recorded branching history - they often use heuristics that forward branches are not taken, backwards branches are taken or something like that.

[matthieum]

@cg909

Code-placement is an implementation detail. Even on the level of LLVM IR they're still about branch weights. Even if it only affects code placement on most architectures the goal still is to optimize for the more likely branch.

And I take exception at "the more likely".

In latency-sensitive fields, the optimizations that developers wish for are those that optimize for the latency of the data-flows that matter, and NOT those that optimize for generic throughput.

As an example, consider busy-polling: most of the time the result of the polling operation is "there is nothing to do" while the case were performance matters is most likely the few times were "there is something to do".

Code-placement is effective at optimizing for the branch that matters, because it ensures that the next instructions to execute are either already in cache, or being pre-fetched as we speak.

An hypothetical instruction to disable the branch predictor, or instruct which branch to predict, would also fall into optimizing the branch that matters -- regardless of likelihood.

Hence I would favor renaming the hint to mirror its effect -- hinting at the compiler which branch matters -- rather than keeping a name that applies only in a (large) subset of cases.

[pdolezal]

For busy waiting Java has Thread::onSpinWait for some time. It is a JIT intrinsic that calms the CPU down. For congested loops with atomics it can bring nice performance boost. I wonder if Rust could have something similar.

[scottmcm]

@pdolezal

It is a JIT intrinsic that calms the CPU down.

Define what "calms" means here.

Rust has hint::spin_loop() and thread::yield_now() and thread::sleep_ms(0) depending on the meaning you're hoping for.

[pdolezal]

The documentation leaves that to the JVM implementation, but most implementations would probably use an equivalent of hint::spin_loop (thanks for the pointer, btw. ı didn't know it).