The tag has no wiki summary.

learn more… | top users | synonyms

25
votes
10answers
5k views

Why are floating point numbers used often in Science/Engineering?

While investigating the accuracy of floating point numbers, I've seen in a few places a statement similar to "float and double are (designed for/used often in) engineering and scientific ...
0
votes
1answer
209 views

How is a floating point number represented in Java?

Which algorithm does java uses to convert floating point number (Ex:0.15625) into binary(0.00101)? Whether java uses normalized form or denormalized format? If I type float ab=0.15625 in Java source ...
9
votes
1answer
215 views

Implied precision of floating-point functions

While reviewing another programmer's implementation of a function to calculate the normal distribution CDF, I made a suggestion to either replace the entire implementation with Python's built-in ...
2
votes
2answers
142 views

How are some IEEE 754 floats printed succinctly?

Using this calculator, I can see how the decimal ".8" and the expression ".7 + .1" have different representations: .8 = 1.1001100110011001100110011001100110011001100110011010 *2-1 .7 + .1 = ...
1
vote
0answers
39 views

Time passage arithmetic explanation

I ported this from http://www.effectgames.com/effect/article.psp.html/joe/Old_School_Color_Cycling_with_HTML5 some time ago. However i'm now wanting to modify it for the purpose of changing it from ...
40
votes
3answers
4k views

Are there numbers that are not representable in base 10 but can be represented in base 2?

C# has the decimal type which is used for numbers that needs exact representation in base 10. For instance, 0.1 cannot be represented in base 2 (e.g. float and double) and will always be an ...
0
votes
4answers
144 views

Do commercial statistical calculators calculate with a high degree of arbitrary precision or just floating point precision? [closed]

Do commercial statistical calculators like Minitab or SAS (and open source like R) perform statistical analysis with very high precision? Or do they just use floating point (or double) precision? I ...
2
votes
3answers
2k views

Why is BigDecimal the best data type for currency? [duplicate]

I was reading this question and the accepted answer says that BigDecimal is the best type for representing currency values. I've also seen several other questions support the use of big decimal. Why ...
4
votes
1answer
286 views

Custom class for monetary amount (money value) instead of BigDecimal

On StackOverflow I found multiple advices to stick with BigDecimal in Java instead of reinventing the wheel. Despite that, I decided to create my custom class, and I would ask if that was a good idea ...
4
votes
3answers
529 views

When do rounding problems become a real problem? Is the least significant digit being one off really a big deal? [duplicate]

If I do .1 + .1 + .1 in Python, I get 0.30000000000000004. (I am not asking about Python in particular, and do not want Python specific answers.) The only problem I can see with this is ...
1
vote
1answer
236 views

Will we go back(?) to fixed-point arithmetic in the near future? [closed]

As far as I recall, this has been the ongoing trend of the past, I'm just a student, so I might be wrong: Long ago: Integer numbers and calculations. Very precise, however cannot present a big ...
22
votes
11answers
3k views

Why do we still use floats? [duplicate]

I understand why floats served a purpose in the past. And I think I can see why they're useful in some simulation examples today. But I think those example are more exceptional than common. So I ...
3
votes
2answers
287 views

Why does IEEE 754 Single Float have 8 bits for exponent but only allow 38 base ten Max Value

Why is it that the IEEE 754 standard provide 8 bits for the exponent but puts a range maximum of 38 on the largest acceptable exponent? Seems kind of ridiculous to have the data storage there but not ...
6
votes
3answers
293 views

Unit (regression) testing scientific algorithms given floating point behavior

I have been working on a project and running into a very difficult problem. The problem can be stated simply as how to unit-test numerical algorithms. However if you just took this simple statement ...
0
votes
1answer
102 views

How can I handle clock hands using doubles that suffers at most O(log n) corrupt bits at nth frame of display?

I presently have a clock app that calculates from scratch at every iteration. This means O(1) corrupt bits in my doubles and heavy object creation and deletion as well. I am wary of running ...
2
votes
4answers
294 views

Generic term for “objects” vs “fundamental types”?

What are the exact terms to call data types with a logic structure (like C structures, C++ or Java objects) versus fundamental data types (like numeric types, characters, booleans...) independently of ...
2
votes
3answers
371 views

Why is float the default in the majority of languages?

In programming languages, for represent numbers you have (mainly) two types: Int(s) and floats. The us of Int is very easy. However floats cause a lot of unexpected surprises: ...
8
votes
2answers
2k views

How Lua handles both integer and float numbers?

As far as I remember myself programming I was taught not to compare floating point numbers for equality. Now, while reading Programming in Lua about Lua number type, I found following: The number ...
2
votes
1answer
1k views

Addition of double's is NOT Equal to Sum of the double as a whole

I am aware of the floating point errors as I had gained some knowledge with my question asked here in SE Floating Point Errors. What I am finding it difficult to understand is, the output of the ...
1
vote
2answers
445 views

Implementing base-10 floating point division

I'm implementing floating-point arithmetic, for a micro-controller which does not support floating point numbers, in either hardware or software. (Software being "written" in a sort of electrical ...
2
votes
7answers
804 views

Is (1/(1/x)) always a perfect round trip?

Is the following guaranteed to return true for all numerical and non-zero values of x? bool IsRoundTrip(double x) { double y = 1 / (1 / x); return x == y; } What conditions would cause a ...