floating point - Rule of thumb to test the equality of two doubles in C#?

Question

Let's say I have some code that does some floating point arithmetic and stores the values in doubles. Because some values can't be represented perfectly in binary, how do I test for equality to a reasonable degree of certainty?

How do I determine what "reasonable" means?

Can double.Epsilon be used in some way?

---

Update

Couple things. As @ho1 pointed out, the documentation for double.Epsilon points out that, when it comes to comparing two doubles for equality, you are probably going to want a value much greater than epsilon. Here is the relevant paragraph from the documentation:

Two apparently equivalent floating-point numbers might not compare equal because of differences in their least significant digits. For example, the C# expression, (double)1/3 == (double)0.33333, does not compare equal because the division operation on the left side has maximum precision while the constant on the right side is precise only to the specified digits. If you create a custom algorithm that determines whether two floating-point numbers can be considered equal, you must use a value that is greater than the Epsilon constant to establish the acceptable absolute margin of difference for the two values to be considered equal. (Typically, that margin of difference is many times greater than Epsilon.) -- http://msdn.microsoft.com/en-us/library/system.double.epsilon.aspx

...but the question is, how many times greater??

In case it would affect your answer, my particular situation involves geometry calculations (such as dot products and cross products using points and vectors). In some cases, you reach different conclusions based on whether A == B, A > B, or A < B, so I'm looking for a good rule of thumb for how to determine the size of the equivalence window.

Answer 1

Using double.Epsilon does NOT necessarily work. double.Epsilon gives the smallest representable value that is greater than zero. However, because of the way that floating point numbers are implemented, they have less precision the further away from zero they are, so checking for a difference of double.Epsilon could fail for two large numbers that are very close to each other.

Details: A base-2 floating point number is represented as a significand - a number between 1 and 2 - multiplied by two raised to some exponent. A double has 52 bits for the fractional portion of the significand plus 11 bits of precision for the exponent. If the exponent is a very large negative value and the significand is 0, then you get values close to double.Epsilon, but if your exponent is big enough, then even a very small difference in two significands' values will result in a value much larger than double.Epsilon.

For a full discussion on how to test two floating point numbers for equality, see "Comparing Floating Point Numbers, 2012 Edition", by Bruce Dawson. To summarize, there are three main methods of comparison:

Use an absolute difference

As in Joel Coehoorn's example, but be very careful to select a value that's of an appropriate magnitude, unlike Joel's example.

Use a relative difference

Something like the following:

if (Math.Abs(a - b) / b <= maxRelativeError)
{
    return true;
}

However, there are complications; you should divide by the larger of the two values, and this function performs poorly for values close to zero unless you also add a check for a maximum absolute difference. See the paper for details.

Using units of last place

Comparison using units of last place (ULPs) means checking the last portion of the significand. (The paper refers to this as "Comparing using integers.") This is a more complicated approach but is very robust. The paper provides source code in C; for C#, you could probably use BitConverter.DoubleToInt64Bits.

In response to your edit

"How many times greater?" This is really a question of your application domain, which is probably why the .NET Framework doesn't provide a default method, but I've had good luck using the ULPs comparison with a max ULPs difference of 4.