Pre-RFC: Dealing with broken floating point

[rkruppe]

So my suggestion: We should look into applications and see what guarantees they require. Then we can see if implementations provide those and what we can do to rectify the problems on those that don’t.

That would be great, but I think there is not enough volunteer time and interest to do this properly, once and for all, in the Rust project. On the other hand, documenting which floating point implementation will be used is relatively little work on our end, and doesn’t make many commitments, but it gives interested third parties enough information to work around most problems. It’s the “80% of the results with 20% of the effort” route. Or more like 40% of the results with 5% of the effort.

To be clear, I would be delighted if Rust could solve even a small percentage of these problems. I’m just not convinced that it is a realistic goal.

[eefriedman]

Transcendental functions are usually implemented purely or mostly in software

We should be able to address issues with transcendental functions by just providing our own correct libm implementation, which would be consistent across platforms. Not particularly high on the priority list, but there isn’t any particular reason we’re using the system libm other than laziness. (IIRC, IEEE 754 says most transcendental functions are supposed to be within 1 ULP, in deference to the table-maker’s dilemma.)

---

target_float seems sort of useless; as a user, I don’t really care what instructions floating-point arithmetic uses, but rather whether I can expect correct results. I mean, sure, neon implies denormals are flushed, but whether vfp does depends on the target ABI, so that isn’t really useful.

We could provide CFG flags for specifically float_80_bit and float_flush_subnormals, but except in obscure cases I’m not sure what we would actually expect the user to do about them. We can’t safely allow the user to actually manipulate the floating-point environment (to, for example, turn flushing subnormals on or off) until LLVM gains support for it.

[cuviper]

just providing our own correct libm implementation

You make this sound trivial!

SLOCCount finds 46,055 lines in glibc/sysdeps/ieee754/ alone, estimating 11.16 Person-Years of development effort. And I’d guess proper floating point math is even harder than average development.

[eefriedman]

“providing our own correct libm implementation” doesn’t mean it has to be written in Rust… we could potentially use https://github.com/JuliaLang/openlibm.

[rkruppe]

whether vfp does depends on the target ABI, so that isn’t really useful.

Which one can test by also consulting target_os and possibly other such cfg values, right?

I’m not sure what we would actually expect the user to do about them. We can’t safely allow the user to actually manipulate the floating-point environment (to, for example, turn flushing subnormals on or off) until LLVM gains support for it.

Yeah, that particular option is not possible. But there are other things one can do with that knowledge:

• Use different algorithms that are more robust to rounding errors, less likely to produce subnormals, etc. (which is what float parsing in libcore would do).
• Adjust epsilons/tolerances because you noticed that your simulation sometimes explodes with EPS = 1e-10 but seems to go smoothly with EPS = 1-e9.
• Turn off unit tests that are bound to fail (e.g. because they test behavior on subnormal inputs).
• Other hacky workarounds.

I’m not expecting people to independently come up with principled solutions to the entirety of “denormals flush to zero”, I just want to help them with applying their ad-hoc workarounds for their specific bugs. In any case, it’s quite possible that feature-based flags like float_80_bit and float_flush_subnormals are more helpful than exposing ever little detail of the target environment.

[lifthrasiir]

I second the use of openlibm. It seems to be the highest-quality libm alternative available under liberal licenses. In the past I have also considered the use of metalibm which can be tweaked to directly generate a Rust code, but its licensing state is unclear and I’m yet to receive a reply to my request to clarify the matter.

[llogiq]

Perhaps we can nick a solution to a part of those problems from the Haskell folks. Herbie is a compiler plugin that converts FP calculations to numerically stable (but slower) code. I’ve already set up an issue at clippy, perhaps we can at least lint for unstable calculations.

That is not to say such calculations are erroneous, but one should think about the error bounds one is willing to accept given certain ranges of inputs, and being shown where those calculations may fail can lead to valuable insight.

[dhardy]

So my suggestion: We should look into applications and see what guarantees they require.

I was working on a project where reproducibility of results across platforms was important. We ended up enforcing use of SSE2 and excluding the very few users with older CPUs (we never targetted non-x86 architectures anyway). For that application at least requiring SSE2 was sufficient and not too disruptive.

[jimuazu]

How about a library that tests at runtime all the various known failures of FP implementations. Then any code which cares about that can check the conformance-status of the CPU it’s running on, and fail early.

[arielb1]

There should be a way of knowing a CPU is not bit-for-bit IEEE754 compliant, for when reproducible results are desired (in that case, the only solution is to refuse to compile, but that’s much better than subtle bit errors).

If a processor flushes denormals to zero, one must also be careful not to accidentally divide by that zero and crash the program.

[rkruppe]

Is Rust code compiled for SSE2 hardware actually bit-for-bit IEEE 754 compliant? I can’t strongly assert one way or another, but I would not be surprised if there are some relatively obscure edge cases that are not covered, even ignoring things like rounding modes that aren’t exposed to Rust programs anyway.

If a processor flushes denormals to zero, one must also be careful not to accidentally divide by that zero and crash the program.

Dividing by 0.0 does not crash anything, it silently produces ±∞ (or NaN). In the default configuration, that is, but like rounding modes this cannot be changed from Rust.

[vks]

See also the longest GCC bug report (AFAIK).

[dhardy]

Well, it might be nice to have a compiler option to do FP operations in software when targeting non-compliant architectures. It might also be nice to have a standard library function to check for known deviations from IEEE 754.

But like the GCC bug report says, this problem is never going to be fixed.

[alexchandel]

This is why we need to provide our own “libm” instead of using the system’s! There’s already an RFC for using openlibm with Rust: https://github.com/rust-lang/rfcs/issues/711.

[jimb]

From that GCC bug:

(5) A radical solution: Find a job/hobby where computers are not used at all.

LOL

[jimb]

@dhardy wrote:

But like the GCC bug report says, this problem is never going to be fixed.

The problem was fixed; see comment 127, and Joseph Myers’ post. (Not that I can blame you for not reading that far!)

I think it’s important to note that GCC bug 323 was actually two separate problems rolled up into one very ugly ball:

• Until SSE2 (Pentium 4, 2001), the Intel 32-bit architecture had no way to do math with IEEE double precision in registers; to get a true double result, you had to spill to memory, which was slow. If you were expecting IEEE double results, you’d be disappointed. But the C standard permitted this excess precision for temporary results, so this would have been your mistake.

• When and whether GCC truncated values to IEEE double depended on the optimizer’s decisions: it simply wasn’t predictable to the programmer at all. It occurred in places not permitted by the C standard. You couldn’t assume that (x/y) == (x/y) was true.

The first problem is something it makes perfect sense for Rust to address: a lot of the fast floating-point support out there is not quite IEEE; if we can tell people when frequently-problematic quirks are present so they can work around them, that’d be great.

The second problem is just flat-out unworkable behavior for a compiler.

[arielb1]

What’s the status of this?

[rkruppe]

I am unsure which option, if any, is good enough to move forward. Also I am still hoping someone can settle this question:

Is Rust code compiled for SSE2 hardware actually bit-for-bit IEEE 754 compliant? I can’t strongly assert one way or another, but I would not be surprised if there are some relatively obscure edge cases that are not covered, even ignoring things like rounding modes that aren’t exposed to Rust programs anyway.

I vaguely recall learning that LLVM always ignores (and thus breaks) signalling NaNs in its optimizations. I don’t have a source though, and I don’t know if the behavior that’s broken is technically part of IEEE 754. Furthermore, even if it’s part of the standard, I don’t know if anyone cares at all. sNaNs are really obscure and you can’t even produce them without either deliberately constructing one through bit fiddling or by setting some FPU flags via inline assembly.

However one should file an issue for the openlibm thing. It sounds pretty uncontroversial since it’s almost no extra work and reduces cross-platform differences (I think libstd has some hacks caused by MSVC’s libm being weird, so those could disappear as well).

[wthrowe]

I vaguely recall learning that LLVM always ignores (and thus breaks) signalling NaNs in its optimizations. I don’t have a source though

LLVM bug 6050 “floating-point operations have side effects” is at least close. It’s about floating point exceptions rather than sNaNs, but I imagine the issues are closely related.

I’ve encountered that issue in another (non-Rust) project where code broke when compiled using clang.

[rkruppe]

So today I learned about cfg(target_feature="..."). I’m kinda surprised that neither I nor anyone else in this thread knew about it, it already exists since July 2015, although it’s unstable. This attribute already covers pretty much everything the hypothetical cfg(target_float="...") would have covered (other than libm stuff which was more of a random side thought), and has the advantage of already existing. It also has the disadvantage of being so naive as to be basically useless (https://github.com/rust-lang/rust/issues/31662), but whatever, that can be fixed.

The cfg(float_is_broken) alternative, on the other hand, seems less and less attractive every time I look at it. @wthrowe describes how LLVM will foil attempts at strict IEEE compatibility, the flag is a very coarse thing anyway, and now the problem it was intended to address already has another solution.

To me, that settles the topic I set out to address with this thread, though this thread has also generated other good ideas, like using openlibm!