c++ - Why does adding 0 to the end of float literal change how it rounds (possible GCC bug)?

Question

I discovered on my x86 VM (32 bit) that the following program:

#include <stdio.h>
void foo (long double x) {
    int y = x;
    printf("(int)%Lf = %d
", x, y);
}
int main () {
    foo(.9999999999999999999728949456878623891498136799780L);
    foo(.999999999999999999972894945687862389149813679978L);
    return 0;
}

Produces the following output:

(int)1.000000 = 1
(int)1.000000 = 0

Ideone also produces this behavior.

What is the compiler doing to allow this to happen?

I found this constant as I was tracking down why the following program didn't produce 0 as I expected (using 19 9s produced the 0 I expected):

int main () {
    long double x = .99999999999999999999L; /* 20 9's */
    int y = x;
    printf("%d
", y);
    return 0;
}

As I tried to compute the value at which the result switches from expected to unexpected, I arrived at the constant this question is about.

Answer 1

Your problem is that long double on your platform has insufficient precision to store the exact value 0.99999999999999999999. This means that the value of that must be converted to a representable value (this conversion happens during translation of your program, not at runtime).

This conversion can generate either the nearest representable value, or the next greater or smaller representable value. The choice is implementation-defined, so your implementation should document which it is using. It seems that your implementation uses x87-style 80bit long double, and is rounding to the nearest value, resulting in a value of 1.0 stored in x.

---

With the assumed format for long double (with 64 mantissa bits), the highest representable number less than 1.0 is, in hexadecimal:

0x0.ffffffffffffffff

The number exactly halfway between this value and the next higher representable number (1.0) is:

0x0.ffffffffffffffff8

Your very long constant 0.9999999999999999999728949456878623891498136799780 is equal to:

0x0.ffffffffffffffff7fffffffffffffffffffffffa1eb2f0b64cf31c113a8ec...

which should obviously be rounded down if rounding to nearest, but you appear to have reached some limit of the floating point representation your compiler is using, or a rounding bug.