Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Background

I hate the way .Net/IEEE-754 handles equality of floating-point numbers (FPNs) (i.e. double, float). It requires the programmer to be prescient with respect to the yet-to-be-determined history of the number, as the programmer must choose a 'reasonable' value of epsilon where frequently no such value can be determined (as the programmer cannot determine ahead of time how many operations, hence roundings, the number will be subject to).

I would like to create a 'better' FPN within .Net. At creation, one would set its initial scientific error (or accept a default; the default being the minimum). When one performed operations on the FPN, those operations would update the scientific error (to account for the effect of rounding). When testing the equality of two FPNs, one could determine if their ranges overlapped (and even the probability that they are the same number).

Question

The code itself is simple, my questions are:

  • How do operations (+ - * / to start with) affect the scientific error of a FPN? Are there formulae for determining this scientific error?
  • I understand that an added complication is the uneven distribution of FPNs. This can obviously be accounted for by scaling the error in proportion to the inherent scientific error of the result; but is there a formula for determining the inherent scientific error present in a given FPN?
share|improve this question
    
You are aware this isn't ".NET's floating point numbers" - it's "IEEE-754 floating point numbers" right? –  Jon Skeet Mar 27 '13 at 7:09
    
@JonSkeet Yeah, I am, but I only hate the way .Net handles them because I'm not working with other languages presently ;-) I flip/flopped between writing ".Net" and "most programming language" a few times. –  Mark Mar 27 '13 at 7:20
    
You do not mean “scientific error”. That term is not used in numerical computing. Perhaps you are thinking of something like scientific notation, which scales, so you actually mean relative error. –  Eric Postpischil Mar 27 '13 at 11:42
    
As one answer has mentioned, interval arithmetic may provide some of what you seek, but it cannot provide the probability that two results are the same number (that is, that they are the results of computations that would produce the same number of performed with exact arithmetic) for several reasons. First, you would need probability distributions of the input values. Second, tracking the probability distributions would be much harder than tracking simple intervals. Third, if the probability distributions are continuous, the probability that two numbers are equal is usually zero. –  Eric Postpischil Mar 27 '13 at 11:45
    
Why do you say the programmer cannot determine ahead of time how many operations a number will be subjected to? Much numerical code uses a number of operations that is determined at the time the code is written. –  Eric Postpischil Mar 27 '13 at 11:47
show 5 more comments

migrated from stackoverflow.com Mar 29 '13 at 10:58

This question came from our site for professional and enthusiast programmers.

2 Answers

up vote 6 down vote accepted

It has already been done, see Interval arithmetic. This type of automatic floating-point error analysis gives you a reliable upper bound on the accumulated rounding error. Perhaps the biggest disadvantage is the Dependency problem which can lead to correct but way too conservative error bounds.

share|improve this answer
1  
Of course, the interval arithmetic version of == usually returns only the interval [0, 0] or [0, 1]. It cannot return [1, 1] unless the two inputs are each a single point. Interval arithmetic could state whether two intervals overlap or not, but we do not know whether that will solve the questioner’s underlying problem since they have not described their actual application. –  Eric Postpischil Mar 27 '13 at 11:53
    
@Ali Thank you, this is simple and makes sense. It addresses the first of my questions (regarding how to calculate the error present after various basic operations), and has the added benefit of catering for asymmetrical error. Any thoughts on how to deal with the uneven distribution of FPNs (i.e. larger-magnitude numbers inherently having more error than smaller-magnitude numbers)? –  Mark Mar 28 '13 at 0:13
1  
@Mark Interval arithmetic will automagically take care of that. –  Ali Mar 28 '13 at 9:10
add comment

Start by reading Hamming's classic book Numerical Methods for Scientists and Engineers. He goes into a great deal of detail on the nature of floating-point numbers and of numbers in science in general. (Interesting tidbit: Floating point numbers are not uniformly distributed, and neither are the fundamental constants of physics, but the the two distributions are similar.)

In any computational problem, you have to have some idea what constitutes a "good" answer. In simplest form: what value of epsilon is appropriate? For example, for problems with distances measured in astronomical units (1 AU = 93 million miles, approximately), epsilon of 1 mile corresponds to 1E-8 relative error. Depending on what you are doing, that may or may not be enough.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.