Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm trying to make floating point number systems a bit more intuitive for myself. There are a few things I am confused about, and I think the best way to clear up my confusions would be for someone could guide me through one question: How many numbers are there in a floating point number system (Given the base, precision, max exponent, and min exponent)?

This is what I am thinking:

I figure that the maximum possible number (realmax) divided by the smallest possible number (realmin) would give all of the possible positive numbers. Part of me also believes I could divide realmax by the smallest increment, machine epsilon (eps), to figure out how many numbers there are. Just looking at the differences in magnitude between eps (10^-16 for IEEE double precision) and realmin (10^-308), however, tells me this isn't true at all. I can't think of an intuition for why!

So the problem I am facing is determining the correct formula for realmax and realmin. The answer I get using my textbook's formulas differ radically from the one I got with wikipedia's formula.

Help!

share|improve this question
    
Is this in relation with a specific programming language or are we just talking hypothetically here? –  ldigas Jan 30 '12 at 0:34
    
Due to the pidgeonhole principle, the amount of possible numbers is directly related to the available bits minus the amount of invalid sequences of bits (which, IIRC, there is only one - negative zero?) (Not an answer because I don't know enough about floating point formats - someone else, feel free to expand this or contradict me) –  Izkata Jan 30 '12 at 0:39
    
Just hypothetical. In class we have discussed that a floating point number system can be described with just four variables: base, precision, max exponent, min exponent. –  user46346 Jan 31 '12 at 5:41
add comment

3 Answers 3

The following is based on my own deduction and I have no proof of its accuracy.

Ultimately, how many bytes does a floating point number consume? A computer can't possibly represent more unique numbers than it can unique bit patterns. For a 64-bit floating point (C# double) there are 2^64 unique values. Note that some combinations give equivilent values. Quoting Wikipedia:

While the exponent (11-bits for C# double) can be positive or negative, in binary formats it is stored as an unsigned number that has a fixed "bias" added to it. Values of all 0s in this field are reserved for the zeros and subnormal numbers, values of all 1s are reserved for the infinities and NaNs.

So this means there's 2^53 combinations that represent infinate or invalid numbers, and 2^53 combinations that represent zero and subnormal numbers. I can't say one way or the other whether there are other bit-combinations that will produce the same number.

2^64 - 2^53 + 3 = 18,437,736,874,454,810,627 unique values (Represents all bit combinations with positive infinity, negative infinity, and not-a-number combinations being condensed to three unique values.)

Read Floating point, Internal representation.

share|improve this answer
1  
Not sure why you're eliminating the subnormal numbers. They're perfectly valid floating point numbers; they're just expressed slightly differently from normal numbers. –  David Wallace Jan 30 '12 at 0:55
    
This makes sense to me and is helpful! However, what I would like to do is represent the total amount using the variables B, P, EMAX, EMIN. These are the base/radix, the precision, and the max and min exponents. For IEEE double precision, I believe B=2, P=53 (or 52?), EMAX=1023, EMIN= -1022. –  user46346 Jan 30 '12 at 1:15
1  
@DavidWallace, I'm not eliminating subnormal numbers. I'm eliminating all variations of infinity and NaN. –  Hand-E-Food Jan 30 '12 at 1:52
    
@HandEFood - My apologies. I need to go away and learn to read. –  David Wallace Jan 30 '12 at 1:55
    
@DavidWallace, no worries. I'm frequently guilty of that. :-) –  Hand-E-Food Jan 30 '12 at 2:26
show 3 more comments

The reason why your intuition about smallest increments doesn't work out is that the increments increase as the numbers get bigger. This is so that very small numbers can be expressed with a higher precision than very big numbers (for which several decimal places probably don't matter). Every time you cross a power of 2, the size of the increment doubles. So the number of floating point numbers between 8 and 16 is the same as the number of floating point numbers between 4 and 8 - they're just spaced twice as far apart. And this, in turn, is the same as the number of floating point numbers between 2 and 4; and also between 1 and 2. Of course, exactly how many this is depends on whether you're using single or double precision.

This pattern keeps going until you get down to the smallest possible exponent. For numbers smaller than this, the increments are all the same.

share|improve this answer
add comment

You're trying to find the granularity of a floating point representation. This is formally called a "Machine Epsilon"

You can compute it with this formula

Machine Epsilon Formula

You then take that and plug it into this formula 1.0(XOR)(epsilon)=1.0. If that holds the Epsilon is correct.

Note that this Epsilon is a measure of the granularity for numbers with an absolute value of 1.0. Epsilon decreases (i.e. more precision) for numbers whose absolute value is less than 1, and increases (less precision) for numbers whose absolute value is greater than 1.

Formulas and such from here.

share|improve this answer
    
I find this a bit confusing. What is XOR? –  user46346 Jan 30 '12 at 1:15
    
Exclusive OR en.wikipedia.org/wiki/Xor –  World Engineer Jan 30 '12 at 1:17
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.