Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

While reviewing another programmer's implementation of a function to calculate the normal distribution CDF, I made a suggestion to either replace the entire implementation with Python's built-in functions or use SciPy, a common scientific library.

Another programmer pointed out that neither math.erfc() nor scipy.stats.norm.cdf() provides any precision guarantees in their documentation. Therefore, I should be more cautious about replacing an approximation algorithm (which was taken from a respected source, and which had documented error bounds).

To be honest, the thought to doubt the accuracy and precision of a built-in or library function had never crossed my mind. After all, I've been calling functions like sin() and sqrt() for years without much thought — why should math.erf() or scipy.stats.norm.cdf() be any different?

But now, I'm concerned. My questions are:

  1. In general, if the documentation makes no special mention, is it implied that these kinds of functions are completely accurate to the last decimal place, within the precision offered by IEEE double-precision floating-point?
  2. Is that true for Python's math.erf() or SciPy's scipy.stats.norm.cdf() in particular? How can you tell?
  3. This man page for sin() says…

    These functions may lose accuracy when their argument is near a multiple of pi or is far from 0.0.

    Why should such caveats exist, when the sine function is periodic and symmetrical? There seems to be a burden placed on the caller to canonicalize the input to obtain optimal accuracy.

    On the other hand, Mozilla's documentation for Math.sin() says nothing about accuracy or precision. Does that mean that it is fully accurate, or is it "common knowledge" that Math.sin() would only be accurate in certain circumstances in JavaScript, like everywhere else?

share|improve this question
1  
FYI regarding question 1: Usually precision guarantees are given in terms of ULP (units in the last place) which refers to the binary digits of the float. –  delnan Aug 8 at 19:23

1 Answer 1

If the documentation makes no special mention, is it implied that these kinds of functions are completely accurate to the last decimal place, within the precision offered by IEEE double-precision floating-point?

I wouldn't make that assumption.

Where I work we deal with telemetry data, and it's common knowledge that two different math libraries can produce two different results, even if they both conform to the IEEE floating point standards. This has implications when you're trying to repeat a calculation and compare two results for equality.

There seems to be a burden placed on the caller to canonicalize the input to obtain optimal accuracy.

That's a fair assessment. What makes it acceptable is that the documentation states that, so that there are no surprises.

On the other hand, Mozilla's documentation ...

I suggest that you try a few calculations in each library or programming language (especially near the boundary areas like sin() near a multiple of pi), and compare them. This should give you a fair idea of what sort of behavior you can expect from each one.

share|improve this answer
1  
Test is a good idea. There are countless times where the documentation says one thing, yet the functions behave another way. And OP want to rely on implicit assumptions that are not even documented. –  Siyuan Ren Aug 9 at 3:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.