The tag has no usage guidance.

learn more… | top users | synonyms

39
votes
7answers
25k views

What causes floating point rounding errors?

I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error. However, I ...
12
votes
3answers
5k views

Handling extremely large numbers in a language which can't?

I'm trying to think about how I would go about doing calculations on extremely large numbers (to infinitum - intergers no floats) if the language construct is incapable of handling numbers larger than ...
10
votes
5answers
3k views

Solutions for floating point rounding errors

In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors. While I understand that floating point is not ...
2
votes
2answers
9k views

Is this a good way to compare two numbers?

If we have double numbers, let's say I want to see if some double parameter is equal to zero that is passed as double: public bool AlmostEqual(double x, double y) { double epsilon = Math.Max(Math....