QSEN #30 – Is it close?

Hi!

How do you test if two numbers are approximately equal, or close to each other? Do you use the math.isclose from Python standard library? Or do you prefer the isclose function from numpy? And, most importantly, are you aware that those two work slightly differently? That’s right, today we want to talk about an interesting case of competing conventions which might cause some bugs that can fly under your radar for a long time!

Checking for approximate equality

If you are used to using either math.isclose or numpy.isclose, you might not be familiar with the other option altogether. For completeness, let’s first recall the signature of both functions. The math.isclose function has the following signature: math.isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0), while numpy.isclose has the following one: numpy.isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False). There are a couple of differences one can spot at first glance:

The names of the arguments for relative and absolute tolerances are named differently. Also, for math.isclose, the tolerances are purely keyword arguments, whereas for numpy.isclose both values can be supplied as positional args.
The default values of tolerances are different.
In numpy, you can choose whether the special NaNs values compare as equal to each other (if you didn’t know that, it might surprise you that NaN is usually considered to not be equal to NaN).

Let’s forget about NaNs for the rest of this newsletter, and only focus on usages that are common to both functions. They have different defaults, but if you explicitly supply the same a and b and same tolerances, would the result always be the same? Perhaps surprisingly, the answer is no. Let’s see what the differences are.

Two formulas for checking approximate equality

When you take a look at documentation of both functions, it becomes clear that the formulas they use are different. For math.isclose, the formula is:

abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)

and for numpy.isclose we have:

abs(a - b) <= (atol + rtol * abs(b)

Let’s break it down:

The formula used by math.isclose is symmetric, meaning that math.isclose(a, b) is equal to math.isclose(b, a). It answers the question: is the difference between a and b smaller than either absolute tolerance or larger of absolute values of those numbers scaled by relative tolerance?
The formula used by numpy.isclose answer the question: is the difference between a and b smaller than sum of absolute tolerance and the absolute value of b scaled by relative tolerance. The formula is asymmetric, because only b is taken into account for comparison with relative tolerance.

Example inputs that’ll give different output for both functions are easy to construct. Let’s start by considering an absolute tolerance of 0 and relative tolerance of 0.1.


import math
import numpy as np

a = 0.0011
b = 0.001

atol = 0.0
rtol = 0.1

print(f"{math.isclose(a, b, abs_tol=atol, rel_tol=rtol)=}")  # True
print(f"{math.isclose(b, a, abs_tol=atol, rel_tol=rtol)=}")  # True
print(f"{np.isclose(a, b, atol=atol, rtol=rtol)=}")          # False
print(f"{np.isclose(b, a, atol=atol, rtol=rtol)=}")          # True

As you can see, depending on the order of inputs, the comparison performed by np.isclose can end up being either True or False. In contrast, the comparison with math.isclose returns True in both cases. Note that it’s not always the case that math.isclose is weaker. For instance, using the following inputs, we obtain True with numpy and False for math:

a = 0.0011
b = 0.001

atol = 0.0001
rtol = 0.05

print(f"{math.isclose(a, b, abs_tol=atol, rel_tol=rtol)=}")  # False
print(f"{math.isclose(b, a, abs_tol=atol, rel_tol=rtol)=}")  # False
print(f"{np.isclose(a, b, atol=atol, rtol=rtol)=}")          # True
print(f"{np.isclose(b, a, atol=atol, rtol=rtol)=}")          # True

Which one should I use then?

Judging the formulas used in isclose functions is beside the point of this newsletter, so unfortunately we cannot tell you which one you should use (although Konrad strongly prefers math.isclose for most uses). The point we are trying to make is that sometimes implementations of ubiquitous, seemingly simple functionallities can substantially differ in unexpected ways. Every time you make a choice between using several possible implementations make sure to stay informed. Read the docs, note down the differences and only then make a conscious choice of what to use.

Have a good one!

Konrad & Michał