The tag has no wiki summary.

learn more… | top users | synonyms

25
votes
6answers
7k views

What causes floating point rounding errors?

I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error. However, I ...
6
votes
2answers
560 views

Solutions for floating point rounding errors

In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors. While I understand that floating point is not ...