The tag has no wiki summary.

learn more… | top users | synonyms

27
votes
6answers
10k views

What causes floating point rounding errors?

I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error. However, I ...
6
votes
2answers
982 views

Solutions for floating point rounding errors

In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors. While I understand that floating point is not ...
1
vote
2answers
2k views

Is this a good way to compare two numbers?

If we have double numbers, let's say I want to see if some double parameter is equal to zero that is passed as double: public bool AlmostEqual(double x, double y) { double epsilon = ...
11
votes
4answers
2k views

Handling extremely large numbers in a language which can't?

I'm trying to think about how I would go about doing calculations on extremely large numbers (to infinitum - intergers no floats) if the language construct is incapable of handling numbers larger than ...